Nanoparticles in Higher-Order Multimodal Imaging

NASA Astrophysics Data System (ADS)

Rieffel, James Ki

Imaging procedures are a cornerstone in our current medical infrastructure. In everything from screening, diagnostics, and treatment, medical imaging is perhaps our greatest tool in evaluating individual health. Recently, there has been tremendous increase in the development of multimodal systems that combine the strengths of complimentary imaging technologies to overcome their independent weaknesses. Clinically, this has manifested in the virtually universal manufacture of combined PET-CT scanners. With this push toward more integrated imaging, new contrast agents with multimodal functionality are needed. Nanoparticle-based systems are ideal candidates based on their unique size, properties, and diversity. In chapter 1, an extensive background on recent multimodal imaging agents capable of enhancing signal or contrast in three or more modalities is presented. Chapter 2 discusses the development and characterization of a nanoparticulate probe with hexamodal imaging functionality. It is my hope that the information contained in this thesis will demonstrate the many benefits of nanoparticles in multimodal imaging, and provide insight into the potential of fully integrated imaging.

Tinnitus Multimodal Imaging

DTIC Science & Technology

2014-10-01

1 AWARD NUMBER: W81XWH-13-1-0494 TITLE: Tinnitus Multimodal Imaging PRINCIPAL INVESTIGATOR...TYPE Annual 3. DATES COVERED 30 Sept 2013 – 29 Oct 2014 4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER Tinnitus Multimodal Imaging...AVAILABILITY STATEMENT Approved for Public Release; Distribution Unlimited 13. SUPPLEMENTARY NOTES 14. ABSTRACT Tinnitus is a common auditory

Multi-Modality Phantom Development

DOE Office of Scientific and Technical Information (OSTI.GOV)

Huber, Jennifer S.; Peng, Qiyu; Moses, William W.

2009-03-20

Multi-modality imaging has an increasing role in the diagnosis and treatment of a large number of diseases, particularly if both functional and anatomical information are acquired and accurately co-registered. Hence, there is a resulting need for multi modality phantoms in order to validate image co-registration and calibrate the imaging systems. We present our PET-ultrasound phantom development, including PET and ultrasound images of a simple prostate phantom. We use agar and gelatin mixed with a radioactive solution. We also present our development of custom multi-modality phantoms that are compatible with PET, transrectal ultrasound (TRUS), MRI and CT imaging. We describe bothmore » our selection of tissue mimicking materials and phantom construction procedures. These custom PET-TRUS-CT-MRI prostate phantoms use agargelatin radioactive mixtures with additional contrast agents and preservatives. We show multi-modality images of these custom prostate phantoms, as well as discuss phantom construction alternatives. Although we are currently focused on prostate imaging, this phantom development is applicable to many multi-modality imaging applications.« less

Photoacoustic-Based Multimodal Nanoprobes: from Constructing to Biological Applications.

PubMed

Gao, Duyang; Yuan, Zhen

2017-01-01

Multimodal nanoprobes have attracted intensive attentions since they can integrate various imaging modalities to obtain complementary merits of single modality. Meanwhile, recent interest in laser-induced photoacoustic imaging is rapidly growing due to its unique advantages in visualizing tissue structure and function with high spatial resolution and satisfactory imaging depth. In this review, we summarize multimodal nanoprobes involving photoacoustic imaging. In particular, we focus on the method to construct multimodal nanoprobes. We have divided the synthetic methods into two types. First, we call it "one for all" concept, which involves intrinsic properties of the element in a single particle. Second, "all in one" concept, which means integrating different functional blocks in one particle. Then, we simply introduce the applications of the multifunctional nanoprobes for in vivo imaging and imaging-guided tumor therapy. At last, we discuss the advantages and disadvantages of the present methods to construct the multimodal nanoprobes and share our viewpoints in this area.

Multimodal Diffuse Optical Imaging

NASA Astrophysics Data System (ADS)

Intes, Xavier; Venugopal, Vivek; Chen, Jin; Azar, Fred S.

Diffuse optical imaging, particularly diffuse optical tomography (DOT), is an emerging clinical modality capable of providing unique functional information, at a relatively low cost, and with nonionizing radiation. Multimodal diffuse optical imaging has enabled a synergistic combination of functional and anatomical information: the quality of DOT reconstructions has been significantly improved by incorporating the structural information derived by the combined anatomical modality. In this chapter, we will review the basic principles of diffuse optical imaging, including instrumentation and reconstruction algorithm design. We will also discuss the approaches for multimodal imaging strategies that integrate DOI with clinically established modalities. The merit of the multimodal imaging approaches is demonstrated in the context of optical mammography, but the techniques described herein can be translated to other clinical scenarios such as brain functional imaging or muscle functional imaging.

Prospective Evaluation of Multimodal Optical Imaging with Automated Image Analysis to Detect Oral Neoplasia In Vivo.

PubMed

Quang, Timothy; Tran, Emily Q; Schwarz, Richard A; Williams, Michelle D; Vigneswaran, Nadarajah; Gillenwater, Ann M; Richards-Kortum, Rebecca

2017-10-01

The 5-year survival rate for patients with oral cancer remains low, in part because diagnosis often occurs at a late stage. Early and accurate identification of oral high-grade dysplasia and cancer can help improve patient outcomes. Multimodal optical imaging is an adjunctive diagnostic technique in which autofluorescence imaging is used to identify high-risk regions within the oral cavity, followed by high-resolution microendoscopy to confirm or rule out the presence of neoplasia. Multimodal optical images were obtained from 206 sites in 100 patients. Histologic diagnosis, either from a punch biopsy or an excised surgical specimen, was used as the gold standard for all sites. Histopathologic diagnoses of moderate dysplasia or worse were considered neoplastic. Images from 92 sites in the first 30 patients were used as a training set to develop automated image analysis methods for identification of neoplasia. Diagnostic performance was evaluated prospectively using images from 114 sites in the remaining 70 patients as a test set. In the training set, multimodal optical imaging with automated image analysis correctly classified 95% of nonneoplastic sites and 94% of neoplastic sites. Among the 56 sites in the test set that were biopsied, multimodal optical imaging correctly classified 100% of nonneoplastic sites and 85% of neoplastic sites. Among the 58 sites in the test set that corresponded to a surgical specimen, multimodal imaging correctly classified 100% of nonneoplastic sites and 61% of neoplastic sites. These findings support the potential of multimodal optical imaging to aid in the early detection of oral cancer. Cancer Prev Res; 10(10); 563-70. ©2017 AACR . ©2017 American Association for Cancer Research.

Multimodality Image Fusion-Guided Procedures: Technique, Accuracy, and Applications

DOE Office of Scientific and Technical Information (OSTI.GOV)

Abi-Jaoudeh, Nadine, E-mail: naj@mail.nih.gov; Kruecker, Jochen, E-mail: jochen.kruecker@philips.com; Kadoury, Samuel, E-mail: samuel.kadoury@polymtl.ca

2012-10-15

Personalized therapies play an increasingly critical role in cancer care: Image guidance with multimodality image fusion facilitates the targeting of specific tissue for tissue characterization and plays a role in drug discovery and optimization of tailored therapies. Positron-emission tomography (PET), magnetic resonance imaging (MRI), and contrast-enhanced computed tomography (CT) may offer additional information not otherwise available to the operator during minimally invasive image-guided procedures, such as biopsy and ablation. With use of multimodality image fusion for image-guided interventions, navigation with advanced modalities does not require the physical presence of the PET, MRI, or CT imaging system. Several commercially available methodsmore » of image-fusion and device navigation are reviewed along with an explanation of common tracking hardware and software. An overview of current clinical applications for multimodality navigation is provided.« less

Multimodal quantitative phase and fluorescence imaging of cell apoptosis

NASA Astrophysics Data System (ADS)

Fu, Xinye; Zuo, Chao; Yan, Hao

2017-06-01

Fluorescence microscopy, utilizing fluorescence labeling, has the capability to observe intercellular changes which transmitted and reflected light microscopy techniques cannot resolve. However, the parts without fluorescence labeling are not imaged. Hence, the processes simultaneously happen in these parts cannot be revealed. Meanwhile, fluorescence imaging is 2D imaging where information in the depth is missing. Therefore the information in labeling parts is also not complete. On the other hand, quantitative phase imaging is capable to image cells in 3D in real time through phase calculation. However, its resolution is limited by the optical diffraction and cannot observe intercellular changes below 200 nanometers. In this work, fluorescence imaging and quantitative phase imaging are combined to build a multimodal imaging system. Such system has the capability to simultaneously observe the detailed intercellular phenomenon and 3D cell morphology. In this study the proposed multimodal imaging system is used to observe the cell behavior in the cell apoptosis. The aim is to highlight the limitations of fluorescence microscopy and to point out the advantages of multimodal quantitative phase and fluorescence imaging. The proposed multimodal quantitative phase imaging could be further applied in cell related biomedical research, such as tumor.

Multimodal imaging of ischemic wounds

NASA Astrophysics Data System (ADS)

Zhang, Shiwu; Gnyawali, Surya; Huang, Jiwei; Liu, Peng; Gordillo, Gayle; Sen, Chandan K.; Xu, Ronald

2012-12-01

The wound healing process involves the reparative phases of inflammation, proliferation, and remodeling. Interrupting any of these phases may result in chronically unhealed wounds, amputation, or even patient death. Quantitative assessment of wound tissue ischemia, perfusion, and inflammation provides critical information for appropriate detection, staging, and treatment of chronic wounds. However, no method is available for noninvasive, simultaneous, and quantitative imaging of these tissue parameters. We integrated hyperspectral, laser speckle, and thermographic imaging modalities into a single setup for multimodal assessment of tissue oxygenation, perfusion, and inflammation characteristics. Advanced algorithms were developed for accurate reconstruction of wound oxygenation and appropriate co-registration between different imaging modalities. The multimodal wound imaging system was validated by an ongoing clinical trials approved by OSU IRB. In the clinical trial, a wound of 3mm in diameter was introduced on a healthy subject's lower extremity and the healing process was serially monitored by the multimodal imaging setup. Our experiments demonstrated the clinical usability of multimodal wound imaging.

The pivotal role of multimodality reporter sensors in drug discovery: from cell based assays to real time molecular imaging.

PubMed

Ray, Pritha

2011-04-01

Development and marketing of new drugs require stringent validation that are expensive and time consuming. Non-invasive multimodality molecular imaging using reporter genes holds great potential to expedite these processes at reduced cost. New generations of smarter molecular imaging strategies such as Split reporter, Bioluminescence resonance energy transfer, Multimodality fusion reporter technologies will further assist to streamline and shorten the drug discovery and developmental process. This review illustrates the importance and potential of molecular imaging using multimodality reporter genes in drug development at preclinical phases.

In vivo multimodal nonlinear optical imaging of mucosal tissue

NASA Astrophysics Data System (ADS)

Sun, Ju; Shilagard, Tuya; Bell, Brent; Motamedi, Massoud; Vargas, Gracie

2004-05-01

We present a multimodal nonlinear imaging approach to elucidate microstructures and spectroscopic features of oral mucosa and submucosa in vivo. The hamster buccal pouch was imaged using 3-D high resolution multiphoton and second harmonic generation microscopy. The multimodal imaging approach enables colocalization and differentiation of prominent known spectroscopic and structural features such as keratin, epithelial cells, and submucosal collagen at various depths in tissue. Visualization of cellular morphology and epithelial thickness are in excellent agreement with histological observations. These results suggest that multimodal nonlinear optical microscopy can be an effective tool for studying the physiology and pathology of mucosal tissue.

SU-E-J-110: A Novel Level Set Active Contour Algorithm for Multimodality Joint Segmentation/Registration Using the Jensen-Rényi Divergence.

PubMed

Markel, D; Naqa, I El; Freeman, C; Vallières, M

2012-06-01

To present a novel joint segmentation/registration for multimodality image-guided and adaptive radiotherapy. A major challenge to this framework is the sensitivity of many segmentation or registration algorithms to noise. Presented is a level set active contour based on the Jensen-Renyi (JR) divergence to achieve improved noise robustness in a multi-modality imaging space. To present a novel joint segmentation/registration for multimodality image-guided and adaptive radiotherapy. A major challenge to this framework is the sensitivity of many segmentation or registration algorithms to noise. Presented is a level set active contour based on the Jensen-Renyi (JR) divergence to achieve improved noise robustness in a multi-modality imaging space. It was found that JR divergence when used for segmentation has an improved robustness to noise compared to using mutual information, or other entropy-based metrics. The MI metric failed at around 2/3 the noise power than the JR divergence. The JR divergence metric is useful for the task of joint segmentation/registration of multimodality images and shows improved results compared entropy based metric. The algorithm can be easily modified to incorporate non-intensity based images, which would allow applications into multi-modality and texture analysis. © 2012 American Association of Physicists in Medicine.

Supervised multiblock sparse multivariable analysis with application to multimodal brain imaging genetics.

PubMed

Kawaguchi, Atsushi; Yamashita, Fumio

2017-10-01

This article proposes a procedure for describing the relationship between high-dimensional data sets, such as multimodal brain images and genetic data. We propose a supervised technique to incorporate the clinical outcome to determine a score, which is a linear combination of variables with hieratical structures to multimodalities. This approach is expected to obtain interpretable and predictive scores. The proposed method was applied to a study of Alzheimer's disease (AD). We propose a diagnostic method for AD that involves using whole-brain magnetic resonance imaging (MRI) and positron emission tomography (PET), and we select effective brain regions for the diagnostic probability and investigate the genome-wide association with the regions using single nucleotide polymorphisms (SNPs). The two-step dimension reduction method, which we previously introduced, was considered applicable to such a study and allows us to partially incorporate the proposed method. We show that the proposed method offers classification functions with feasibility and reasonable prediction accuracy based on the receiver operating characteristic (ROC) analysis and reasonable regions of the brain and genomes. Our simulation study based on the synthetic structured data set showed that the proposed method outperformed the original method and provided the characteristic for the supervised feature. © The Author 2017. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Ridge-branch-based blood vessel detection algorithm for multimodal retinal images

NASA Astrophysics Data System (ADS)

Li, Y.; Hutchings, N.; Knighton, R. W.; Gregori, G.; Lujan, B. J.; Flanagan, J. G.

2009-02-01

Automatic detection of retinal blood vessels is important to medical diagnoses and imaging. With the development of imaging technologies, various modals of retinal images are available. Few of currently published algorithms are applied to multimodal retinal images. Besides, the performance of algorithms with pathologies is expected to be improved. The purpose of this paper is to propose an automatic Ridge-Branch-Based (RBB) detection algorithm of blood vessel centerlines and blood vessels for multimodal retinal images (color fundus photographs, fluorescein angiograms, fundus autofluorescence images, SLO fundus images and OCT fundus images, for example). Ridges, which can be considered as centerlines of vessel-like patterns, are first extracted. The method uses the connective branching information of image ridges: if ridge pixels are connected, they are more likely to be in the same class, vessel ridge pixels or non-vessel ridge pixels. Thanks to the good distinguishing ability of the designed "Segment-Based Ridge Features", the classifier and its parameters can be easily adapted to multimodal retinal images without ground truth training. We present thorough experimental results on SLO images, color fundus photograph database and other multimodal retinal images, as well as comparison between other published algorithms. Results showed that the RBB algorithm achieved a good performance.

NaGdF4:Nd3+/Yb3+ Nanoparticles as Multimodal Imaging Agents

NASA Astrophysics Data System (ADS)

Pedraza, Francisco; Rightsell, Chris; Kumar, Ga; Giuliani, Jason; Monton, Car; Sardar, Dhiraj

Medical imaging is a fundamental tool used for the diagnosis of numerous ailments. Each imaging modality has unique advantages; however, they possess intrinsic limitations. Some of which include low spatial resolution, sensitivity, penetration depth, and radiation damage. To circumvent this problem, the combination of imaging modalities, or multimodal imaging, has been proposed, such as Near Infrared Fluorescence imaging (NIRF) and Magnetic Resonance Imaging (MRI). Combining individual advantages, specificity and selectivity of NIRF with the deep penetration and high spatial resolution of MRI, it is possible to circumvent their shortcomings for a more robust imaging technique. In addition, both imaging modalities are very safe and minimally invasive. Fluorescent nanoparticles, such as NaGdF4:Nd3 +/Yb3 +, are excellent candidates for NIRF/MRI multimodal imaging. The dopants, Nd and Yb, absorb and emit within the biological window; where near infrared light is less attenuated by soft tissue. This results in less tissue damage and deeper tissue penetration making it a viable candidate in biological imaging. In addition, the inclusion of Gd results in paramagnetic properties, allowing their use as contrast agents in multimodal imaging. The work presented will include crystallographic results, as well as full optical and magnetic characterization to determine the nanoparticle's viability in multimodal imaging.

Developing single-laser sources for multimodal coherent anti-Stokes Raman scattering microscopy

NASA Astrophysics Data System (ADS)

Pegoraro, Adrian Frank

Coherent anti-Stokes Raman scattering (CARS) microscopy has developed rapidly and is opening the door to new types of experiments. This work describes the development of new laser sources for CARS microscopy and their use for different applications. It is specifically focused on multimodal nonlinear optical microscopy—the simultaneous combination of different imaging techniques. This allows us to address a diverse range of applications, such as the study of biomaterials, fluid inclusions, atherosclerosis, hepatitis C infection in cells, and ice formation in cells. For these applications new laser sources are developed that allow for practical multimodal imaging. For example, it is shown that using a single Ti:sapphire oscillator with a photonic crystal fiber, it is possible to develop a versatile multimodal imaging system using optimally chirped laser pulses. This system can perform simultaneous two photon excited fluorescence, second harmonic generation, and CARS microscopy. The versatility of the system is further demonstrated by showing that it is possible to probe different Raman modes using CARS microscopy simply by changing a time delay between the excitation beams. Using optimally chirped pulses also enables further simplification of the laser system required by using a single fiber laser combined with nonlinear optical fibers to perform effective multimodal imaging. While these sources are useful for practical multimodal imaging, it is believed that for further improvements in CARS microscopy sensitivity, new excitation schemes are necessary. This has led to the design of a new, high power, extended cavity oscillator that should be capable of implementing new excitation schemes for CARS microscopy as well as other techniques. Our interest in multimodal imaging has led us to other areas of research as well. For example, a fiber-coupling scheme for signal collection in the forward direction is demonstrated that allows for fluorescence lifetime imaging without significant temporal distortion. Also highlighted is an imaging artifact that is unique to CARS microscopy that can alter image interpretation, especially when using multimodal imaging. By combining expertise in nonlinear optics, laser development, fiber optics, and microscopy, we have developed systems and techniques that will be of benefit for multimodal CARS microscopy.

Mobile, Multi-modal, Label-Free Imaging Probe Analysis of Choroidal Oximetry and Retinal Hypoxia

DTIC Science & Technology

2015-10-01

eyes and image choroidal vessels/capillaries using CARS intravital microscopy Subtask 3: Measure oxy-hemoglobin levels in PBI test and control eyes...AWARD NUMBER: W81XWH-14-1-0537 TITLE: Mobile, Multi-modal, Label-Free Imaging Probe Analysis of Choroidal Oximetry and Retinal Hypoxia...4. TITLE AND SUBTITLE Mobile, Multimodal, Label-Free Imaging Probe Analysis of Choroidal Oximetry and Retinal Hypoxia 5a. CONTRACT NUMBER W81XWH

Radioactive Nanomaterials for Multimodality Imaging

PubMed Central

Chen, Daiqin; Dougherty, Casey A.; Yang, Dongzhi; Wu, Hongwei; Hong, Hao

2016-01-01

Nuclear imaging techniques, including primarily positron emission tomography (PET) and single-photon emission computed tomography (SPECT), can provide quantitative information for a biological event in vivo with ultra-high sensitivity, however, the comparatively low spatial resolution is their major limitation in clinical application. By convergence of nuclear imaging with other imaging modalities like computed tomography (CT), magnetic resonance imaging (MRI) and optical imaging, the hybrid imaging platforms can overcome the limitations from each individual imaging technique. Possessing versatile chemical linking ability and good cargo-loading capacity, radioactive nanomaterials can serve as ideal imaging contrast agents. In this review, we provide a brief overview about current state-of-the-art applications of radioactive nanomaterials in the circumstances of multimodality imaging. We present strategies for incorporation of radioisotope(s) into nanomaterials along with applications of radioactive nanomaterials in multimodal imaging. Advantages and limitations of radioactive nanomaterials for multimodal imaging applications are discussed. Finally, a future perspective of possible radioactive nanomaterial utilization is presented for improving diagnosis and patient management in a variety of diseases. PMID:27227167

MO-DE-202-03: Image-Guided Surgery and Interventions in the Advanced Multimodality Image-Guided Operating (AMIGO) Suite

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kapur, T.

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guidedmore » neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504 Disclosure and CoI: IGI Technologies, small-business partner on the grants.« less

Robust Multimodal Dictionary Learning

PubMed Central

Cao, Tian; Jojic, Vladimir; Modla, Shannon; Powell, Debbie; Czymmek, Kirk; Niethammer, Marc

2014-01-01

We propose a robust multimodal dictionary learning method for multimodal images. Joint dictionary learning for both modalities may be impaired by lack of correspondence between image modalities in training data, for example due to areas of low quality in one of the modalities. Dictionaries learned with such non-corresponding data will induce uncertainty about image representation. In this paper, we propose a probabilistic model that accounts for image areas that are poorly corresponding between the image modalities. We cast the problem of learning a dictionary in presence of problematic image patches as a likelihood maximization problem and solve it with a variant of the EM algorithm. Our algorithm iterates identification of poorly corresponding patches and re-finements of the dictionary. We tested our method on synthetic and real data. We show improvements in image prediction quality and alignment accuracy when using the method for multimodal image registration. PMID:24505674

Multimodal partial volume correction: Application to [11C]PIB PET/MRI myelin imaging in multiple sclerosis.

PubMed

Grecchi, Elisabetta; Veronese, Mattia; Bodini, Benedetta; García-Lorenzo, Daniel; Battaglini, Marco; Stankoff, Bruno; Turkheimer, Federico E

2017-12-01

The [ 11 C]PIB PET tracer, originally developed for amyloid imaging, has been recently repurposed to quantify demyelination and remyelination in multiple sclerosis (MS). Myelin PET imaging, however, is limited by its low resolution that deteriorates the quantification accuracy of white matter (WM) lesions. Here, we introduce a novel partial volume correction (PVC) method called Multiresolution-Multimodal Resolution-Recovery (MM-RR), which uses the wavelet transform and a synergistic statistical model to exploit MRI structural images to improve the resolution of [ 11 C]PIB PET myelin imaging. MM-RR performance was tested on a phantom acquisition and in a dataset comprising [ 11 C]PIB PET and MR T1- and T2-weighted images of 8 healthy controls and 20 MS patients. For the control group, the MM-RR PET images showed an average increase of 5.7% in WM uptake while the grey-matter (GM) uptake remained constant, resulting in +31% WM/GM contrast. Furthermore, MM-RR PET binding maps correlated significantly with the mRNA expressions of the most represented proteins in the myelin sheath (R 2  = 0.57 ± 0.09). In the patient group, MM-RR PET images showed sharper lesion contours and significant improvement in normal-appearing tissue/WM-lesion contrast compared to standard PET (contrast improvement > +40%). These results were consistent with MM-RR performances in phantom experiments.

Multimodal fusion of brain imaging data: A key to finding the missing link(s) in complex mental illness

PubMed Central

Calhoun, Vince D; Sui, Jing

2016-01-01

It is becoming increasingly clear that combining multi-modal brain imaging data is able to provide more information for individual subjects by exploiting the rich multimodal information that exists. However, the number of studies that do true multimodal fusion (i.e. capitalizing on joint information among modalities) is still remarkably small given the known benefits. In part, this is because multi-modal studies require broader expertise in collecting, analyzing, and interpreting the results than do unimodal studies. In this paper, we start by introducing the basic reasons why multimodal data fusion is important and what it can do, and importantly how it can help us avoid wrong conclusions and help compensate for imperfect brain imaging studies. We also discuss the challenges that need to be confronted for such approaches to be more widely applied by the community. We then provide a review of the diverse studies that have used multimodal data fusion (primarily focused on psychosis) as well as provide an introduction to some of the existing analytic approaches. Finally, we discuss some up-and-coming approaches to multi-modal fusion including deep learning and multimodal classification which show considerable promise. Our conclusion is that multimodal data fusion is rapidly growing, but it is still underutilized. The complexity of the human brain coupled with the incomplete measurement provided by existing imaging technology makes multimodal fusion essential in order to mitigate against misdirection and hopefully provide a key to finding the missing link(s) in complex mental illness. PMID:27347565

Multimodal fusion of brain imaging data: A key to finding the missing link(s) in complex mental illness.

PubMed

Calhoun, Vince D; Sui, Jing

2016-05-01

It is becoming increasingly clear that combining multi-modal brain imaging data is able to provide more information for individual subjects by exploiting the rich multimodal information that exists. However, the number of studies that do true multimodal fusion (i.e. capitalizing on joint information among modalities) is still remarkably small given the known benefits. In part, this is because multi-modal studies require broader expertise in collecting, analyzing, and interpreting the results than do unimodal studies. In this paper, we start by introducing the basic reasons why multimodal data fusion is important and what it can do, and importantly how it can help us avoid wrong conclusions and help compensate for imperfect brain imaging studies. We also discuss the challenges that need to be confronted for such approaches to be more widely applied by the community. We then provide a review of the diverse studies that have used multimodal data fusion (primarily focused on psychosis) as well as provide an introduction to some of the existing analytic approaches. Finally, we discuss some up-and-coming approaches to multi-modal fusion including deep learning and multimodal classification which show considerable promise. Our conclusion is that multimodal data fusion is rapidly growing, but it is still underutilized. The complexity of the human brain coupled with the incomplete measurement provided by existing imaging technology makes multimodal fusion essential in order to mitigate against misdirection and hopefully provide a key to finding the missing link(s) in complex mental illness.

Fluorescence Imaging Topography Scanning System for intraoperative multimodal imaging

PubMed Central

Quang, Tri T.; Kim, Hye-Yeong; Bao, Forrest Sheng; Papay, Francis A.; Edwards, W. Barry; Liu, Yang

2017-01-01

Fluorescence imaging is a powerful technique with diverse applications in intraoperative settings. Visualization of three dimensional (3D) structures and depth assessment of lesions, however, are oftentimes limited in planar fluorescence imaging systems. In this study, a novel Fluorescence Imaging Topography Scanning (FITS) system has been developed, which offers color reflectance imaging, fluorescence imaging and surface topography scanning capabilities. The system is compact and portable, and thus suitable for deployment in the operating room without disturbing the surgical flow. For system performance, parameters including near infrared fluorescence detection limit, contrast transfer functions and topography depth resolution were characterized. The developed system was tested in chicken tissues ex vivo with simulated tumors for intraoperative imaging. We subsequently conducted in vivo multimodal imaging of sentinel lymph nodes in mice using FITS and PET/CT. The PET/CT/optical multimodal images were co-registered and conveniently presented to users to guide surgeries. Our results show that the developed system can facilitate multimodal intraoperative imaging. PMID:28437441

Microscopy with multimode fibers

NASA Astrophysics Data System (ADS)

Moser, Christophe; Papadopoulos, Ioannis; Farahi, Salma; Psaltis, Demetri

2013-04-01

Microscopes are usually thought of comprising imaging elements such as objectives and eye-piece lenses. A different type of microscope, used for endoscopy, consists of waveguiding elements such as fiber bundles, where each fiber in the bundle transports the light corresponding to one pixel in the image. Recently a new type of microscope has emerged that exploits the large number of propagating modes in a single multimode fiber. We have successfully produced fluorescence images of neural cells with sub-micrometer resolution via a 200 micrometer core multimode fiber. The method for achieving imaging consists of using digital phase conjugation to reproduce a focal spot at the tip of the multimode fiber. The image is formed by scanning the focal spot digitally and collecting the fluorescence point by point.

Landmark Image Retrieval by Jointing Feature Refinement and Multimodal Classifier Learning.

PubMed

Zhang, Xiaoming; Wang, Senzhang; Li, Zhoujun; Ma, Shuai; Xiaoming Zhang; Senzhang Wang; Zhoujun Li; Shuai Ma; Ma, Shuai; Zhang, Xiaoming; Wang, Senzhang; Li, Zhoujun

2018-06-01

Landmark retrieval is to return a set of images with their landmarks similar to those of the query images. Existing studies on landmark retrieval focus on exploiting the geometries of landmarks for visual similarity matches. However, the visual content of social images is of large diversity in many landmarks, and also some images share common patterns over different landmarks. On the other side, it has been observed that social images usually contain multimodal contents, i.e., visual content and text tags, and each landmark has the unique characteristic of both visual content and text content. Therefore, the approaches based on similarity matching may not be effective in this environment. In this paper, we investigate whether the geographical correlation among the visual content and the text content could be exploited for landmark retrieval. In particular, we propose an effective multimodal landmark classification paradigm to leverage the multimodal contents of social image for landmark retrieval, which integrates feature refinement and landmark classifier with multimodal contents by a joint model. The geo-tagged images are automatically labeled for classifier learning. Visual features are refined based on low rank matrix recovery, and multimodal classification combined with group sparse is learned from the automatically labeled images. Finally, candidate images are ranked by combining classification result and semantic consistence measuring between the visual content and text content. Experiments on real-world datasets demonstrate the superiority of the proposed approach as compared to existing methods.

Supervised learning based multimodal MRI brain tumour segmentation using texture features from supervoxels.

PubMed

Soltaninejad, Mohammadreza; Yang, Guang; Lambrou, Tryphon; Allinson, Nigel; Jones, Timothy L; Barrick, Thomas R; Howe, Franklyn A; Ye, Xujiong

2018-04-01

Accurate segmentation of brain tumour in magnetic resonance images (MRI) is a difficult task due to various tumour types. Using information and features from multimodal MRI including structural MRI and isotropic (p) and anisotropic (q) components derived from the diffusion tensor imaging (DTI) may result in a more accurate analysis of brain images. We propose a novel 3D supervoxel based learning method for segmentation of tumour in multimodal MRI brain images (conventional MRI and DTI). Supervoxels are generated using the information across the multimodal MRI dataset. For each supervoxel, a variety of features including histograms of texton descriptor, calculated using a set of Gabor filters with different sizes and orientations, and first order intensity statistical features are extracted. Those features are fed into a random forests (RF) classifier to classify each supervoxel into tumour core, oedema or healthy brain tissue. The method is evaluated on two datasets: 1) Our clinical dataset: 11 multimodal images of patients and 2) BRATS 2013 clinical dataset: 30 multimodal images. For our clinical dataset, the average detection sensitivity of tumour (including tumour core and oedema) using multimodal MRI is 86% with balanced error rate (BER) 7%; while the Dice score for automatic tumour segmentation against ground truth is 0.84. The corresponding results of the BRATS 2013 dataset are 96%, 2% and 0.89, respectively. The method demonstrates promising results in the segmentation of brain tumour. Adding features from multimodal MRI images can largely increase the segmentation accuracy. The method provides a close match to expert delineation across all tumour grades, leading to a faster and more reproducible method of brain tumour detection and delineation to aid patient management. Copyright © 2018 Elsevier B.V. All rights reserved.

MO-DE-202-04: Multimodality Image-Guided Surgery and Intervention: For the Rest of Us

DOE Office of Scientific and Technical Information (OSTI.GOV)

Shekhar, R.

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guidedmore » neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504 Disclosure and CoI: IGI Technologies, small-business partner on the grants.« less

Tinnitus Multimodal Imaging

DTIC Science & Technology

2015-10-01

AWARD NUMBER: W81XWH-13-1-0494 TITLE: Tinnitus Multimodal Imaging PRINCIPAL INVESTIGATOR: Steven Wan Cheung CONTRACTING ORGANIZATION...NUMBER W81XWH-13-1-0494 Tinnitus Multimodal Imaging 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER Steven W. Cheung...13. SUPPLEMENTARY NOTES 14. ABSTRACT Tinnitus is a common auditory perceptual disorder whose neural substrates are under intense debate. This project

Multimodal Imaging of the Normal Eye.

PubMed

Kawali, Ankush; Pichi, Francesco; Avadhani, Kavitha; Invernizzi, Alessandro; Hashimoto, Yuki; Mahendradas, Padmamalini

2017-10-01

Multimodal imaging is the concept of "bundling" images obtained from various imaging modalities, viz., fundus photograph, fundus autofluorescence imaging, infrared (IR) imaging, simultaneous fluorescein and indocyanine angiography, optical coherence tomography (OCT), and, more recently, OCT angiography. Each modality has its pros and cons as well as its limitations. Combination of multiple imaging techniques will overcome their individual weaknesses and give a comprehensive picture. Such approach helps in accurate localization of a lesion and understanding the pathology in posterior segment. It is important to know imaging of normal eye before one starts evaluating pathology. This article describes multimodal imaging modalities in detail and discusses healthy eye features as seen on various imaging modalities mentioned above.

A novel automated method for doing registration and 3D reconstruction from multi-modal RGB/IR image sequences

NASA Astrophysics Data System (ADS)

Kirby, Richard; Whitaker, Ross

2016-09-01

In recent years, the use of multi-modal camera rigs consisting of an RGB sensor and an infrared (IR) sensor have become increasingly popular for use in surveillance and robotics applications. The advantages of using multi-modal camera rigs include improved foreground/background segmentation, wider range of lighting conditions under which the system works, and richer information (e.g. visible light and heat signature) for target identification. However, the traditional computer vision method of mapping pairs of images using pixel intensities or image features is often not possible with an RGB/IR image pair. We introduce a novel method to overcome the lack of common features in RGB/IR image pairs by using a variational methods optimization algorithm to map the optical flow fields computed from different wavelength images. This results in the alignment of the flow fields, which in turn produce correspondences similar to those found in a stereo RGB/RGB camera rig using pixel intensities or image features. In addition to aligning the different wavelength images, these correspondences are used to generate dense disparity and depth maps. We obtain accuracies similar to other multi-modal image alignment methodologies as long as the scene contains sufficient depth variations, although a direct comparison is not possible because of the lack of standard image sets from moving multi-modal camera rigs. We test our method on synthetic optical flow fields and on real image sequences that we created with a multi-modal binocular stereo RGB/IR camera rig. We determine our method's accuracy by comparing against a ground truth.

Fast segmentation and high-quality three-dimensional volume mesh creation from medical images for diffuse optical tomography

NASA Astrophysics Data System (ADS)

Jermyn, Michael; Ghadyani, Hamid; Mastanduno, Michael A.; Turner, Wes; Davis, Scott C.; Dehghani, Hamid; Pogue, Brian W.

2013-08-01

Multimodal approaches that combine near-infrared (NIR) and conventional imaging modalities have been shown to improve optical parameter estimation dramatically and thus represent a prevailing trend in NIR imaging. These approaches typically involve applying anatomical templates from magnetic resonance imaging/computed tomography/ultrasound images to guide the recovery of optical parameters. However, merging these data sets using current technology requires multiple software packages, substantial expertise, significant time-commitment, and often results in unacceptably poor mesh quality for optical image reconstruction, a reality that represents a significant roadblock for translational research of multimodal NIR imaging. This work addresses these challenges directly by introducing automated digital imaging and communications in medicine image stack segmentation and a new one-click three-dimensional mesh generator optimized for multimodal NIR imaging, and combining these capabilities into a single software package (available for free download) with a streamlined workflow. Image processing time and mesh quality benchmarks were examined for four common multimodal NIR use-cases (breast, brain, pancreas, and small animal) and were compared to a commercial image processing package. Applying these tools resulted in a fivefold decrease in image processing time and 62% improvement in minimum mesh quality, in the absence of extra mesh postprocessing. These capabilities represent a significant step toward enabling translational multimodal NIR research for both expert and nonexpert users in an open-source platform.

In-situ Multimodal Imaging and Spectroscopy of Mg Electrodeposition at Electrode-Electrolyte Interfaces

NASA Astrophysics Data System (ADS)

Wu, Yimin A.; Yin, Zuwei; Farmand, Maryam; Yu, Young-Sang; Shapiro, David A.; Liao, Hong-Gang; Liang, Wen-I.; Chu, Ying-Hao; Zheng, Haimei

2017-02-01

We report the study of Mg cathodic electrochemical deposition on Ti and Au electrode using a multimodal approach by examining the sample area in-situ using liquid cell transmission electron microscopy (TEM), scanning transmission X-ray microscopy (STXM) and X-ray absorption spectroscopy (XAS). Magnesium Aluminum Chloride Complex was synthesized and utilized as electrolyte, where non-reversible features during in situ charging-discharging cycles were observed. During charging, a uniform Mg film was deposited on the electrode, which is consistent with the intrinsic non-dendritic nature of Mg deposition in Mg ion batteries. The Mg thin film was not dissolvable during the following discharge process. We found that such Mg thin film is hexacoordinated Mg compounds by in-situ STXM and XAS. This study provides insights on the non-reversibility issue and failure mechanism of Mg ion batteries. Also, our method provides a novel generic method to understand the in situ battery chemistry without any further sample processing, which can preserve the original nature of battery materials or electrodeposited materials. This multimodal in situ imaging and spectroscopy provides many opportunities to attack complex problems that span orders of magnitude in length and time scale, which can be applied to a broad range of the energy storage systems.

In-situ Multimodal Imaging and Spectroscopy of Mg Electrodeposition at Electrode-Electrolyte Interfaces

DOE Office of Scientific and Technical Information (OSTI.GOV)

Wu, Yimin A.; Yin, Zuwei; Farmand, Maryam

We report the study of Mg cathodic electrochemical deposition on Ti and Au electrode using a multimodal approach by examining the sample area in-situ using liquid cell transmission electron microscopy (TEM), scanning transmission X-ray microscopy (STXM) and X-ray absorption spectroscopy (XAS). Magnesium Aluminum Chloride Complex was synthesized and utilized as electrolyte, where non-reversible features during in situ charging-discharging cycles were observed. During charging, a uniform Mg film was deposited on the electrode, which is consistent with the intrinsic non-dendritic nature of Mg deposition in Mg ion batteries. The Mg thin film was not dissolvable during the following discharge process. Wemore » found that such Mg thin film is hexacoordinated Mg compounds by in-situ STXM and XAS. This study provides insights on the non-reversibility issue and failure mechanism of Mg ion batteries. Also, our method provides a novel generic method to understand the in situ battery chemistry without any further sample processing, which can preserve the original nature of battery materials or electrodeposited materials. This multimodal in situ imaging and spectroscopy provides many opportunities to attack complex problems that span orders of magnitude in length and time scale, which can be applied to a broad range of the energy storage systems.« less

In-situ Multimodal Imaging and Spectroscopy of Mg Electrodeposition at Electrode-Electrolyte Interfaces

DOE PAGES

Wu, Yimin A.; Yin, Zuwei; Farmand, Maryam; ...

2017-02-10

We report the study of Mg cathodic electrochemical deposition on Ti and Au electrode using a multimodal approach by examining the sample area in-situ using liquid cell transmission electron microscopy (TEM), scanning transmission X-ray microscopy (STXM) and X-ray absorption spectroscopy (XAS). Magnesium Aluminum Chloride Complex was synthesized and utilized as electrolyte, where non-reversible features during in situ charging-discharging cycles were observed. During charging, a uniform Mg film was deposited on the electrode, which is consistent with the intrinsic non-dendritic nature of Mg deposition in Mg ion batteries. The Mg thin film was not dissolvable during the following discharge process. Wemore » found that such Mg thin film is hexacoordinated Mg compounds by in-situ STXM and XAS. This study provides insights on the non-reversibility issue and failure mechanism of Mg ion batteries. Also, our method provides a novel generic method to understand the in situ battery chemistry without any further sample processing, which can preserve the original nature of battery materials or electrodeposited materials. This multimodal in situ imaging and spectroscopy provides many opportunities to attack complex problems that span orders of magnitude in length and time scale, which can be applied to a broad range of the energy storage systems.« less

In-situ Multimodal Imaging and Spectroscopy of Mg Electrodeposition at Electrode-Electrolyte Interfaces.

PubMed

Wu, Yimin A; Yin, Zuwei; Farmand, Maryam; Yu, Young-Sang; Shapiro, David A; Liao, Hong-Gang; Liang, Wen-I; Chu, Ying-Hao; Zheng, Haimei

2017-02-10

We report the study of Mg cathodic electrochemical deposition on Ti and Au electrode using a multimodal approach by examining the sample area in-situ using liquid cell transmission electron microscopy (TEM), scanning transmission X-ray microscopy (STXM) and X-ray absorption spectroscopy (XAS). Magnesium Aluminum Chloride Complex was synthesized and utilized as electrolyte, where non-reversible features during in situ charging-discharging cycles were observed. During charging, a uniform Mg film was deposited on the electrode, which is consistent with the intrinsic non-dendritic nature of Mg deposition in Mg ion batteries. The Mg thin film was not dissolvable during the following discharge process. We found that such Mg thin film is hexacoordinated Mg compounds by in-situ STXM and XAS. This study provides insights on the non-reversibility issue and failure mechanism of Mg ion batteries. Also, our method provides a novel generic method to understand the in situ battery chemistry without any further sample processing, which can preserve the original nature of battery materials or electrodeposited materials. This multimodal in situ imaging and spectroscopy provides many opportunities to attack complex problems that span orders of magnitude in length and time scale, which can be applied to a broad range of the energy storage systems.

In-situ Multimodal Imaging and Spectroscopy of Mg Electrodeposition at Electrode-Electrolyte Interfaces

PubMed Central

Wu, Yimin A.; Yin, Zuwei; Farmand, Maryam; Yu, Young-Sang; Shapiro, David A.; Liao, Hong-Gang; Liang, Wen-I; Chu, Ying-Hao; Zheng, Haimei

2017-01-01

We report the study of Mg cathodic electrochemical deposition on Ti and Au electrode using a multimodal approach by examining the sample area in-situ using liquid cell transmission electron microscopy (TEM), scanning transmission X-ray microscopy (STXM) and X-ray absorption spectroscopy (XAS). Magnesium Aluminum Chloride Complex was synthesized and utilized as electrolyte, where non-reversible features during in situ charging-discharging cycles were observed. During charging, a uniform Mg film was deposited on the electrode, which is consistent with the intrinsic non-dendritic nature of Mg deposition in Mg ion batteries. The Mg thin film was not dissolvable during the following discharge process. We found that such Mg thin film is hexacoordinated Mg compounds by in-situ STXM and XAS. This study provides insights on the non-reversibility issue and failure mechanism of Mg ion batteries. Also, our method provides a novel generic method to understand the in situ battery chemistry without any further sample processing, which can preserve the original nature of battery materials or electrodeposited materials. This multimodal in situ imaging and spectroscopy provides many opportunities to attack complex problems that span orders of magnitude in length and time scale, which can be applied to a broad range of the energy storage systems. PMID:28186175

Anisotropic polyvinyl alcohol hydrogel phantom for shear wave elastography in fibrous biological soft tissue: a multimodality characterization

NASA Astrophysics Data System (ADS)

Chatelin, Simon; Bernal, Miguel; Deffieux, Thomas; Papadacci, Clément; Flaud, Patrice; Nahas, Amir; Boccara, Claude; Gennisson, Jean-Luc; Tanter, Mickael; Pernot, Mathieu

2014-11-01

Shear wave elastography imaging techniques provide quantitative measurement of soft tissues elastic properties. Tendons, muscles and cerebral tissues are composed of fibers, which induce a strong anisotropic effect on the mechanical behavior. Currently, these tissues cannot be accurately represented by existing elastography phantoms. Recently, a novel approach for orthotropic hydrogel mimicking soft tissues has been developed (Millon et al 2006 J. Biomed. Mater. Res. B 305-11). The mechanical anisotropy is induced in a polyvinyl alcohol (PVA) cryogel by stretching the physical crosslinks of the polymeric chains while undergoing freeze/thaw cycles. In the present study we propose an original multimodality imaging characterization of this new transverse isotropic (TI) PVA hydrogel. Multiple properties were investigated using a large variety of techniques at different scales compared with an isotropic PVA hydrogel undergoing similar imaging and rheology protocols. The anisotropic mechanical (dynamic and static) properties were studied using supersonic shear wave imaging technique, full-field optical coherence tomography (FFOCT) strain imaging and classical linear rheometry using dynamic mechanical analysis. The anisotropic optical and ultrasonic spatial coherence properties were measured by FFOCT volumetric imaging and backscatter tensor imaging, respectively. Correlation of mechanical and optical properties demonstrates the complementarity of these techniques for the study of anisotropy on a multi-scale range as well as the potential of this TI phantom as fibrous tissue-mimicking phantom for shear wave elastographic applications.

Multimodal Image Registration through Simultaneous Segmentation.

PubMed

Aganj, Iman; Fischl, Bruce

2017-11-01

Multimodal image registration facilitates the combination of complementary information from images acquired with different modalities. Most existing methods require computation of the joint histogram of the images, while some perform joint segmentation and registration in alternate iterations. In this work, we introduce a new non-information-theoretical method for pairwise multimodal image registration, in which the error of segmentation - using both images - is considered as the registration cost function. We empirically evaluate our method via rigid registration of multi-contrast brain magnetic resonance images, and demonstrate an often higher registration accuracy in the results produced by the proposed technique, compared to those by several existing methods.

Multimodal hard x-ray imaging with resolution approaching 10 nm for studies in material science

NASA Astrophysics Data System (ADS)

Yan, Hanfei; Bouet, Nathalie; Zhou, Juan; Huang, Xiaojing; Nazaretski, Evgeny; Xu, Weihe; Cocco, Alex P.; Chiu, Wilson K. S.; Brinkman, Kyle S.; Chu, Yong S.

2018-03-01

We report multimodal scanning hard x-ray imaging with spatial resolution approaching 10 nm and its application to contemporary studies in the field of material science. The high spatial resolution is achieved by focusing hard x-rays with two crossed multilayer Laue lenses and raster-scanning a sample with respect to the nanofocusing optics. Various techniques are used to characterize and verify the achieved focus size and imaging resolution. The multimodal imaging is realized by utilizing simultaneously absorption-, phase-, and fluorescence-contrast mechanisms. The combination of high spatial resolution and multimodal imaging enables a comprehensive study of a sample on a very fine length scale. In this work, the unique multimodal imaging capability was used to investigate a mixed ionic-electronic conducting ceramic-based membrane material employed in solid oxide fuel cells and membrane separations (compound of Ce0.8Gd0.2O2‑x and CoFe2O4) which revealed the existence of an emergent material phase and quantified the chemical complexity at the nanoscale.

Towards a Compact Fiber Laser for Multimodal Imaging

NASA Astrophysics Data System (ADS)

Nie, Bai; Saytashev, Ilyas; Dantus, Marcos

We report on multimodal depth-resolved imaging of unstained living Drosophila Melanogaster larva using sub-50 fs pulses centered at 1060 nm wavelength. Both second harmonic and third harmonic generation imaging modalities are demonstrated.

Towards a compact fiber laser for multimodal imaging

NASA Astrophysics Data System (ADS)

Nie, Bai; Saytashev, Ilyas; Dantus, Marcos

2014-03-01

We report on multimodal depth-resolved imaging of unstained living Drosophila Melanogaster larva using sub-50 fs pulses centered at 1060 nm wavelength. Both second harmonic and third harmonic generation imaging modalities are demonstrated.

A multimodal image sensor system for identifying water stress in grapevines

NASA Astrophysics Data System (ADS)

Zhao, Yong; Zhang, Qin; Li, Minzan; Shao, Yongni; Zhou, Jianfeng; Sun, Hong

2012-11-01

Water stress is one of the most common limitations of fruit growth. Water is the most limiting resource for crop growth. In grapevines, as well as in other fruit crops, fruit quality benefits from a certain level of water deficit which facilitates to balance vegetative and reproductive growth and the flow of carbohydrates to reproductive structures. A multi-modal sensor system was designed to measure the reflectance signature of grape plant surfaces and identify different water stress levels in this paper. The multi-modal sensor system was equipped with one 3CCD camera (three channels in R, G, and IR). The multi-modal sensor can capture and analyze grape canopy from its reflectance features, and identify the different water stress levels. This research aims at solving the aforementioned problems. The core technology of this multi-modal sensor system could further be used as a decision support system that combines multi-modal sensory data to improve plant stress detection and identify the causes of stress. The images were taken by multi-modal sensor which could output images in spectral bands of near-infrared, green and red channel. Based on the analysis of the acquired images, color features based on color space and reflectance features based on image process method were calculated. The results showed that these parameters had the potential as water stress indicators. More experiments and analysis are needed to validate the conclusion.

A hybrid multimodal non-rigid registration of MR images based on diffeomorphic demons.

PubMed

Lu, Huanxiang; Cattin, Philippe C; Reyes, Mauricio

2010-01-01

In this paper we present a novel hybrid approach for multimodal medical image registration based on diffeomorphic demons. Diffeomorphic demons have proven to be a robust and efficient way for intensity-based image registration. A very recent extension even allows to use mutual information (MI) as a similarity measure to registration multimodal images. However, due to the intensity correspondence uncertainty existing in some anatomical parts, it is difficult for a purely intensity-based algorithm to solve the registration problem. Therefore, we propose to combine the resulting transformations from both intensity-based and landmark-based methods for multimodal non-rigid registration based on diffeomorphic demons. Several experiments on different types of MR images were conducted, for which we show that a better anatomical correspondence between the images can be obtained using the hybrid approach than using either intensity information or landmarks alone.

Multimodal imaging of cutaneous wound tissue

NASA Astrophysics Data System (ADS)

Zhang, Shiwu; Gnyawali, Surya; Huang, Jiwei; Ren, Wenqi; Gordillo, Gayle; Sen, Chandan K.; Xu, Ronald

2015-01-01

Quantitative assessment of wound tissue ischemia, perfusion, and inflammation provides critical information for appropriate detection, staging, and treatment of chronic wounds. However, few methods are available for simultaneous assessment of these tissue parameters in a noninvasive and quantitative fashion. We integrated hyperspectral, laser speckle, and thermographic imaging modalities in a single-experimental setup for multimodal assessment of tissue oxygenation, perfusion, and inflammation characteristics. Algorithms were developed for appropriate coregistration between wound images acquired by different imaging modalities at different times. The multimodal wound imaging system was validated in an occlusion experiment, where oxygenation and perfusion maps of a healthy subject's upper extremity were continuously monitored during a postocclusive reactive hyperemia procedure and compared with standard measurements. The system was also tested in a clinical trial where a wound of three millimeters in diameter was introduced on a healthy subject's lower extremity and the healing process was continuously monitored. Our in vivo experiments demonstrated the clinical feasibility of multimodal cutaneous wound imaging.

MULTIMODAL IMAGING OF SYPHILITIC MULTIFOCAL RETINITIS.

PubMed

Curi, Andre L; Sarraf, David; Cunningham, Emmett T

2015-01-01

To describe multimodal imaging of syphilitic multifocal retinitis. Observational case series. Two patients developed multifocal retinitis after treatment of unrecognized syphilitic uveitis with systemic corticosteroids in the absence of appropriate antibiotic therapy. Multimodal imaging localized the foci of retinitis within the retina in contrast to superficial retinal precipitates that accumulate on the surface of the retina in eyes with untreated syphilitic uveitis. Although the retinitis resolved after treatment with systemic penicillin in both cases, vision remained poor in the patient with multifocal retinitis involving the macula. Treatment of unrecognized syphilitic uveitis with corticosteroids in the absence of antitreponemal treatment can lead to the development of multifocal retinitis. Multimodal imaging, and optical coherence tomography in particular, can be used to distinguish multifocal retinitis from superficial retinal precipitates or accumulations.

Multimodal Imaging and Spatial Analysis of Ebola Retinal Lesions in 14 Survivors of Ebola Virus Disease.

PubMed

Steptoe, Paul J; Momorie, Fayiah; Fornah, Alimamy D; Komba, Sahr P; Emsley, Elizabeth; Scott, Janet T; Harding, Simon P; Vandy, Matthew J; Sahr, Foday; Beare, Nicholas A V; Semple, Malcolm G

2018-05-03

Differentiation between Ebola retinal lesions and other retinal pathologies in West Africa is important, and the pathogenesis of Ebola retinal disease remains poorly understood. To describe the appearance of Ebola virus disease (EVD) retinal lesions using multimodal imaging to enable inferences on potential pathogenesis. This prospective case series study was carried out at 34 Military Hospital in Freetown, Sierra Leone. Ophthalmological images were analyzed from 14 consecutively identified survivors of EVD of Sierra Leonean origin who had identified Ebola retinal lesions. Multimodal imaging findings including ultra-widefield scanning laser ophthalmoscopy, fundus autofluorescence, swept-source optical coherence tomography (OCT), Humphrey visual field analysis, and spatial analysis. The 14 study participants had a mean (SD) age of 37.1 (8.8) years; 6 (43%) were women. A total of 141 Ebola retinal lesions were observed in 22 of 27 eyes (81%) of these 14 survivors on ultra-widefield imaging. Of these, 41 lesions (29.1%) were accessible to OCT imaging. Retinal lesions were predominantly nonpigmented with a pale-gray appearance. Peripapillary lesions exhibited variable curvatures in keeping with the retinal nerve fiber layer projections. All lesions respected the horizontal raphe and spared the fovea. The OCT imaging demonstrated a V-shaped hyperreflectivity of the outer nuclear layer overlying discontinuities of the ellipsoid zone and interdigitation zone in the smaller lesions. Larger lesions caused a collapse of the retinal layers and loss of retinal thickness. Lesion shapes were variable, but sharp angulations were characteristic. Perilesional areas of dark without pressure (thinned ellipsoid zone hyporeflectivity) accompanied 125 of the 141 lesions (88.7%) to varying extents. We demonstrate OCT evidence of localized pathological changes at the level of the photoreceptors in small lesions among survivors of EVD with retinal lesions. The relevance of associated areas of dark without pressure remains undetermined.

Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking.

PubMed

Yu, Jun; Yang, Xiaokang; Gao, Fei; Tao, Dacheng

2017-12-01

How do we retrieve images accurately? Also, how do we rank a group of images precisely and efficiently for specific queries? These problems are critical for researchers and engineers to generate a novel image searching engine. First, it is important to obtain an appropriate description that effectively represent the images. In this paper, multimodal features are considered for describing images. The images unique properties are reflected by visual features, which are correlated to each other. However, semantic gaps always exist between images visual features and semantics. Therefore, we utilize click feature to reduce the semantic gap. The second key issue is learning an appropriate distance metric to combine these multimodal features. This paper develops a novel deep multimodal distance metric learning (Deep-MDML) method. A structured ranking model is adopted to utilize both visual and click features in distance metric learning (DML). Specifically, images and their related ranking results are first collected to form the training set. Multimodal features, including click and visual features, are collected with these images. Next, a group of autoencoders is applied to obtain initially a distance metric in different visual spaces, and an MDML method is used to assign optimal weights for different modalities. Next, we conduct alternating optimization to train the ranking model, which is used for the ranking of new queries with click features. Compared with existing image ranking methods, the proposed method adopts a new ranking model to use multimodal features, including click features and visual features in DML. We operated experiments to analyze the proposed Deep-MDML in two benchmark data sets, and the results validate the effects of the method.

Multimodality cardiac imaging at IRCCS Policlinico San Donato: a new interdisciplinary vision.

PubMed

Lombardi, Massimo; Secchi, Francesco; Pluchinotta, Francesca R; Castelvecchio, Serenella; Montericcio, Vincenzo; Camporeale, Antonia; Bandera, Francesco

2016-04-28

Multimodality imaging is the efficient integration of various methods of cardiovascular imaging to improve the ability to diagnose, guide therapy, or predict outcome. This approach implies both the availability of different technologies in a single unit and the presence of dedicated staff with cardiologic and radiologic background and certified competence in more than one imaging technique. Interaction with clinical practice and existence of research programmes and educational activities are pivotal for the success of this model. The aim of this paper is to describe the multimodality cardiac imaging programme recently started at San Donato Hospital.

A multimodal parallel architecture: A cognitive framework for multimodal interactions.

PubMed

Cohn, Neil

2016-01-01

Human communication is naturally multimodal, and substantial focus has examined the semantic correspondences in speech-gesture and text-image relationships. However, visual narratives, like those in comics, provide an interesting challenge to multimodal communication because the words and/or images can guide the overall meaning, and both modalities can appear in complicated "grammatical" sequences: sentences use a syntactic structure and sequential images use a narrative structure. These dual structures create complexity beyond those typically addressed by theories of multimodality where only a single form uses combinatorial structure, and also poses challenges for models of the linguistic system that focus on single modalities. This paper outlines a broad theoretical framework for multimodal interactions by expanding on Jackendoff's (2002) parallel architecture for language. Multimodal interactions are characterized in terms of their component cognitive structures: whether a particular modality (verbal, bodily, visual) is present, whether it uses a grammatical structure (syntax, narrative), and whether it "dominates" the semantics of the overall expression. Altogether, this approach integrates multimodal interactions into an existing framework of language and cognition, and characterizes interactions between varying complexity in the verbal, bodily, and graphic domains. The resulting theoretical model presents an expanded consideration of the boundaries of the "linguistic" system and its involvement in multimodal interactions, with a framework that can benefit research on corpus analyses, experimentation, and the educational benefits of multimodality. Copyright © 2015.

Design and demonstration of multimodal optical scanning microscopy for confocal and two-photon imaging

NASA Astrophysics Data System (ADS)

Chun, Wanhee; Do, Dukho; Gweon, Dae-Gab

2013-01-01

We developed a multimodal microscopy based on an optical scanning system in order to obtain diverse optical information of the same area of a sample. Multimodal imaging researches have mostly depended on a commercial microscope platform, easy to use but restrictive to extend imaging modalities. In this work, the beam scanning optics, especially including a relay lens, was customized to transfer broadband (400-1000 nm) lights to a sample without any optical error or loss. The customized scanning optics guarantees the best performances of imaging techniques utilizing the lights within the design wavelength. Confocal reflection, confocal fluorescence, and two-photon excitation fluorescence images were obtained, through respective implemented imaging channels, to demonstrate imaging feasibility for near-UV, visible, near-IR continuous light, and pulsed light in the scanning optics. The imaging performances for spatial resolution and image contrast were verified experimentally; the results were satisfactory in comparison with theoretical results. The advantages of customization, containing low cost, outstanding combining ability and diverse applications, will contribute to vitalize multimodal imaging researches.

Dye-enhanced multimodal confocal imaging as a novel approach to intraoperative diagnosis of brain tumors.

PubMed

Snuderl, Matija; Wirth, Dennis; Sheth, Sameer A; Bourne, Sarah K; Kwon, Churl-Su; Ancukiewicz, Marek; Curry, William T; Frosch, Matthew P; Yaroslavsky, Anna N

2013-01-01

Intraoperative diagnosis plays an important role in accurate sampling of brain tumors, limiting the number of biopsies required and improving the distinction between brain and tumor. The goal of this study was to evaluate dye-enhanced multimodal confocal imaging for discriminating gliomas from nonglial brain tumors and from normal brain tissue for diagnostic use. We investigated a total of 37 samples including glioma (13), meningioma (7), metastatic tumors (9) and normal brain removed for nontumoral indications (8). Tissue was stained in 0.05 mg/mL aqueous solution of methylene blue (MB) for 2-5 minutes and multimodal confocal images were acquired using a custom-built microscope. After imaging, tissue was formalin fixed and paraffin embedded for standard neuropathologic evaluation. Thirteen pathologists provided diagnoses based on the multimodal confocal images. The investigated tumor types exhibited distinctive and complimentary characteristics in both the reflectance and fluorescence responses. Images showed distinct morphological features similar to standard histology. Pathologists were able to distinguish gliomas from normal brain tissue and nonglial brain tumors, and to render diagnoses from the images in a manner comparable to haematoxylin and eosin (H&E) slides. These results confirm the feasibility of multimodal confocal imaging for intravital intraoperative diagnosis. © 2012 The Authors; Brain Pathology © 2012 International Society of Neuropathology.

Image-guided plasma therapy of cutaneous wound

NASA Astrophysics Data System (ADS)

Zhang, Zhiwu; Ren, Wenqi; Yu, Zelin; Zhang, Shiwu; Yue, Ting; Xu, Ronald

2014-02-01

The wound healing process involves the reparative phases of inflammation, proliferation, and remodeling. Interrupting any of these phases may result in chronically unhealed wounds, amputation, or even patient death. Despite the clinical significance in chronic wound management, no effective methods have been developed for quantitative image-guided treatment. We integrated a multimodal imaging system with a cold atmospheric plasma probe for image-guided treatment of chronic wound. Multimodal imaging system offers a non-invasive, painless, simultaneous and quantitative assessment of cutaneous wound healing. Cold atmospheric plasma accelerates the wound healing process through many mechanisms including decontamination, coagulation and stimulation of the wound healing. The therapeutic effect of cold atmospheric plasma is studied in vivo under the guidance of a multimodal imaging system. Cutaneous wounds are created on the dorsal skin of the nude mice. During the healing process, the sample wound is treated by cold atmospheric plasma at different controlled dosage, while the control wound is healed naturally. The multimodal imaging system integrating a multispectral imaging module and a laser speckle imaging module is used to collect the information of cutaneous tissue oxygenation (i.e. oxygen saturation, StO2) and blood perfusion simultaneously to assess and guide the plasma therapy. Our preliminary tests show that cold atmospheric plasma in combination with multimodal imaging guidance has the potential to facilitate the healing of chronic wounds.

Improving In Vivo High-Resolution CT Imaging of the Tumour Vasculature in Xenograft Mouse Models through Reduction of Motion and Bone-Streak Artefacts

PubMed Central

Kersemans, Veerle; Kannan, Pavitra; Beech, John S.; Bates, Russell; Irving, Benjamin; Gilchrist, Stuart; Allen, Philip D.; Thompson, James; Kinchesh, Paul; Casteleyn, Christophe; Schnabel, Julia; Partridge, Mike; Muschel, Ruth J.; Smart, Sean C.

2015-01-01

Introduction Preclinical in vivo CT is commonly used to visualise vessels at a macroscopic scale. However, it is prone to many artefacts which can degrade the quality of CT images significantly. Although some artefacts can be partially corrected for during image processing, they are best avoided during acquisition. Here, a novel imaging cradle and tumour holder was designed to maximise CT resolution. This approach was used to improve preclinical in vivo imaging of the tumour vasculature. Procedures A custom built cradle containing a tumour holder was developed and fix-mounted to the CT system gantry to avoid artefacts arising from scanner vibrations and out-of-field sample positioning. The tumour holder separated the tumour from bones along the axis of rotation of the CT scanner to avoid bone-streaking. It also kept the tumour stationary and insensitive to respiratory motion. System performance was evaluated in terms of tumour immobilisation and reduction of motion and bone artefacts. Pre- and post-contrast CT followed by sequential DCE-MRI of the tumour vasculature in xenograft transplanted mice was performed to confirm vessel patency and demonstrate the multimodal capacity of the new cradle. Vessel characteristics such as diameter, and branching were quantified. Results Image artefacts originating from bones and out-of-field sample positioning were avoided whilst those resulting from motions were reduced significantly, thereby maximising the resolution that can be achieved with CT imaging in vivo. Tumour vessels ≥ 77 μm could be resolved and blood flow to the tumour remained functional. The diameter of each tumour vessel was determined and plotted as histograms and vessel branching maps were created. Multimodal imaging using this cradle assembly was preserved and demonstrated. Conclusions The presented imaging workflow minimised image artefacts arising from scanner induced vibrations, respiratory motion and radiopaque structures and enabled in vivo CT imaging and quantitative analysis of the tumour vasculature at higher resolution than was possible before. Moreover, it can be applied in a multimodal setting, therefore combining anatomical and dynamic information. PMID:26046526

MO-DE-202-00: Image-Guided Interventions: Advances in Intraoperative Imaging, Guidance, and An Emerging Role for Medical Physics in Surgery

DOE Office of Scientific and Technical Information (OSTI.GOV)

NONE

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guidedmore » neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504 Disclosure and CoI: IGI Technologies, small-business partner on the grants.« less

MO-DE-202-02: Advances in Image Registration and Reconstruction for Image-Guided Neurosurgery

DOE Office of Scientific and Technical Information (OSTI.GOV)

Siewerdsen, J.

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guidedmore » neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504 Disclosure and CoI: IGI Technologies, small-business partner on the grants.« less

A simultaneous multimodal imaging system for tissue functional parameters

NASA Astrophysics Data System (ADS)

Ren, Wenqi; Zhang, Zhiwu; Wu, Qiang; Zhang, Shiwu; Xu, Ronald

2014-02-01

Simultaneous and quantitative assessment of skin functional characteristics in different modalities will facilitate diagnosis and therapy in many clinical applications such as wound healing. However, many existing clinical practices and multimodal imaging systems are subjective, qualitative, sequential for multimodal data collection, and need co-registration between different modalities. To overcome these limitations, we developed a multimodal imaging system for quantitative, non-invasive, and simultaneous imaging of cutaneous tissue oxygenation and blood perfusion parameters. The imaging system integrated multispectral and laser speckle imaging technologies into one experimental setup. A Labview interface was developed for equipment control, synchronization, and image acquisition. Advanced algorithms based on a wide gap second derivative reflectometry and laser speckle contrast analysis (LASCA) were developed for accurate reconstruction of tissue oxygenation and blood perfusion respectively. Quantitative calibration experiments and a new style of skinsimulating phantom were designed to verify the accuracy and reliability of the imaging system. The experimental results were compared with a Moor tissue oxygenation and perfusion monitor. For In vivo testing, a post-occlusion reactive hyperemia (PORH) procedure in human subject and an ongoing wound healing monitoring experiment using dorsal skinfold chamber models were conducted to validate the usability of our system for dynamic detection of oxygenation and perfusion parameters. In this study, we have not only setup an advanced multimodal imaging system for cutaneous tissue oxygenation and perfusion parameters but also elucidated its potential for wound healing assessment in clinical practice.

Gelatin-based Hydrogel Degradation and Tissue Interaction in vivo: Insights from Multimodal Preclinical Imaging in Immunocompetent Nude Mice.

PubMed

Tondera, Christoph; Hauser, Sandra; Krüger-Genge, Anne; Jung, Friedrich; Neffe, Axel T; Lendlein, Andreas; Klopfleisch, Robert; Steinbach, Jörg; Neuber, Christin; Pietzsch, Jens

2016-01-01

Hydrogels based on gelatin have evolved as promising multifunctional biomaterials. Gelatin is crosslinked with lysine diisocyanate ethyl ester (LDI) and the molar ratio of gelatin and LDI in the starting material mixture determines elastic properties of the resulting hydrogel. In order to investigate the clinical potential of these biopolymers, hydrogels with different ratios of gelatin and diisocyanate (3-fold (G10_LNCO3) and 8-fold (G10_LNCO8) molar excess of isocyanate groups) were subcutaneously implanted in mice (uni- or bilateral implantation). Degradation and biomaterial-tissue-interaction were investigated in vivo (MRI, optical imaging, PET) and ex vivo (autoradiography, histology, serum analysis). Multimodal imaging revealed that the number of covalent net points correlates well with degradation time, which allows for targeted modification of hydrogels based on properties of the tissue to be replaced. Importantly, the degradation time was also dependent on the number of implants per animal. Despite local mechanisms of tissue remodeling no adverse tissue responses could be observed neither locally nor systemically. Finally, this preclinical investigation in immunocompetent mice clearly demonstrated a complete restoration of the original healthy tissue.

MO-DE-202-01: Image-Guided Focused Ultrasound Surgery and Therapy

DOE Office of Scientific and Technical Information (OSTI.GOV)

Farahani, K.

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guidedmore » neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504 Disclosure and CoI: IGI Technologies, small-business partner on the grants.« less

Optical/MRI Multimodality Molecular Imaging

NASA Astrophysics Data System (ADS)

Ma, Lixin; Smith, Charles; Yu, Ping

2007-03-01

Multimodality molecular imaging that combines anatomical and functional information has shown promise in development of tumor-targeted pharmaceuticals for cancer detection or therapy. We present a new multimodality imaging technique that combines fluorescence molecular tomography (FMT) and magnetic resonance imaging (MRI) for in vivo molecular imaging of preclinical tumor models. Unlike other optical/MRI systems, the new molecular imaging system uses parallel phase acquisition based on heterodyne principle. The system has a higher accuracy of phase measurements, reduced noise bandwidth, and an efficient modulation of the fluorescence diffuse density waves. Fluorescent Bombesin probes were developed for targeting breast cancer cells and prostate cancer cells. Tissue phantom and small animal experiments were performed for calibration of the imaging system and validation of the targeting probes.

Development of computational small animal models and their applications in preclinical imaging and therapy research.

PubMed

Xie, Tianwu; Zaidi, Habib

2016-01-01

The development of multimodality preclinical imaging techniques and the rapid growth of realistic computer simulation tools have promoted the construction and application of computational laboratory animal models in preclinical research. Since the early 1990s, over 120 realistic computational animal models have been reported in the literature and used as surrogates to characterize the anatomy of actual animals for the simulation of preclinical studies involving the use of bioluminescence tomography, fluorescence molecular tomography, positron emission tomography, single-photon emission computed tomography, microcomputed tomography, magnetic resonance imaging, and optical imaging. Other applications include electromagnetic field simulation, ionizing and nonionizing radiation dosimetry, and the development and evaluation of new methodologies for multimodality image coregistration, segmentation, and reconstruction of small animal images. This paper provides a comprehensive review of the history and fundamental technologies used for the development of computational small animal models with a particular focus on their application in preclinical imaging as well as nonionizing and ionizing radiation dosimetry calculations. An overview of the overall process involved in the design of these models, including the fundamental elements used for the construction of different types of computational models, the identification of original anatomical data, the simulation tools used for solving various computational problems, and the applications of computational animal models in preclinical research. The authors also analyze the characteristics of categories of computational models (stylized, voxel-based, and boundary representation) and discuss the technical challenges faced at the present time as well as research needs in the future.

An understanding of modified release matrix tablets behavior during drug dissolution as the key for prediction of pharmaceutical product performance - case study of multimodal characterization of quetiapine fumarate tablets.

PubMed

Kulinowski, Piotr; Woyna-Orlewicz, Krzysztof; Rappen, Gerd-Martin; Haznar-Garbacz, Dorota; Węglarz, Władysław P; Dorożyński, Przemysław P

2015-04-30

Motivation for the study was the lack of dedicated and effective research and development (R&D) in vitro methods for oral, generic, modified release formulations. The purpose of the research was to assess multimodal in vitro methodology for further bioequivalence study risk minimization. Principal results of the study are as follows: (i) Pharmaceutically equivalent quetiapine fumarate extended release dosage form of Seroquel XR was developed using a quality by design/design of experiment (QbD/DoE) paradigm. (ii) The developed formulation was then compared with originator using X-ray microtomography, magnetic resonance imaging and texture analysis. Despite similarity in terms of compendial dissolution test, developed and original dosage forms differed in micro/meso structure and consequently in mechanical properties. (iii) These differences were found to be the key factors of failure of biorelevant dissolution test using the stress dissolution apparatus. Major conclusions are as follows: (i) Imaging methods allow to assess internal features of the hydrating extended release matrix and together with the stress dissolution test allow to rationalize the design of generic formulations at the in vitro level. (ii) Technological impact on formulation properties e.g., on pore formation in hydrating matrices cannot be overlooked when designing modified release dosage forms. Copyright © 2015 Elsevier B.V. All rights reserved.

Medical Image Retrieval: A Multimodal Approach

PubMed Central

Cao, Yu; Steffey, Shawn; He, Jianbiao; Xiao, Degui; Tao, Cui; Chen, Ping; Müller, Henning

2014-01-01

Medical imaging is becoming a vital component of war on cancer. Tremendous amounts of medical image data are captured and recorded in a digital format during cancer care and cancer research. Facing such an unprecedented volume of image data with heterogeneous image modalities, it is necessary to develop effective and efficient content-based medical image retrieval systems for cancer clinical practice and research. While substantial progress has been made in different areas of content-based image retrieval (CBIR) research, direct applications of existing CBIR techniques to the medical images produced unsatisfactory results, because of the unique characteristics of medical images. In this paper, we develop a new multimodal medical image retrieval approach based on the recent advances in the statistical graphic model and deep learning. Specifically, we first investigate a new extended probabilistic Latent Semantic Analysis model to integrate the visual and textual information from medical images to bridge the semantic gap. We then develop a new deep Boltzmann machine-based multimodal learning model to learn the joint density model from multimodal information in order to derive the missing modality. Experimental results with large volume of real-world medical images have shown that our new approach is a promising solution for the next-generation medical imaging indexing and retrieval system. PMID:26309389

Tinnitus Multimodal Imaging

DTIC Science & Technology

2016-12-01

images were segmented into gray and white matter images and spatially normalized to the MNI template (3 mm isotropic voxels) using the DARTEL toolbox in...AWARD NUMBER: W81XWH-13-1-0494 TITLE: Tinnitus Multimodal Imaging PRINCIPAL INVESTIGATOR: Steven Wan Cheung CONTRACTING ORGANIZATION... Medical Research and Materiel Command Fort Detrick, Maryland 21702-5012 DISTRIBUTION STATEMENT: Approved for Public Release; Distribution Unlimited

Melanoma detection using smartphone and multimode hyperspectral imaging

NASA Astrophysics Data System (ADS)

MacKinnon, Nicholas; Vasefi, Fartash; Booth, Nicholas; Farkas, Daniel L.

2016-04-01

This project's goal is to determine how to effectively implement a technology continuum from a low cost, remotely deployable imaging device to a more sophisticated multimode imaging system within a standard clinical practice. In this work a smartphone is used in conjunction with an optical attachment to capture cross-polarized and collinear color images of a nevus that are analyzed to quantify chromophore distribution. The nevus is also imaged by a multimode hyperspectral system, our proprietary SkinSpect™ device. Relative accuracy and biological plausibility of the two systems algorithms are compared to assess aspects of feasibility of in-home or primary care practitioner smartphone screening prior to rigorous clinical analysis via the SkinSpect.

Simultaneous multimodal ophthalmic imaging using swept-source spectrally encoded scanning laser ophthalmoscopy and optical coherence tomography

PubMed Central

Malone, Joseph D.; El-Haddad, Mohamed T.; Bozic, Ivan; Tye, Logan A.; Majeau, Lucas; Godbout, Nicolas; Rollins, Andrew M.; Boudoux, Caroline; Joos, Karen M.; Patel, Shriji N.; Tao, Yuankai K.

2016-01-01

Scanning laser ophthalmoscopy (SLO) benefits diagnostic imaging and therapeutic guidance by allowing for high-speed en face imaging of retinal structures. When combined with optical coherence tomography (OCT), SLO enables real-time aiming and retinal tracking and provides complementary information for post-acquisition volumetric co-registration, bulk motion compensation, and averaging. However, multimodality SLO-OCT systems generally require dedicated light sources, scanners, relay optics, detectors, and additional digitization and synchronization electronics, which increase system complexity. Here, we present a multimodal ophthalmic imaging system using swept-source spectrally encoded scanning laser ophthalmoscopy and optical coherence tomography (SS-SESLO-OCT) for in vivo human retinal imaging. SESLO reduces the complexity of en face imaging systems by multiplexing spatial positions as a function of wavelength. SESLO image quality benefited from single-mode illumination and multimode collection through a prototype double-clad fiber coupler, which optimized scattered light throughput and reduce speckle contrast while maintaining lateral resolution. Using a shared 1060 nm swept-source, shared scanner and imaging optics, and a shared dual-channel high-speed digitizer, we acquired inherently co-registered en face retinal images and OCT cross-sections simultaneously at 200 frames-per-second. PMID:28101411

Label-free imaging of acanthamoeba using multimodal nonlinear optical microscopy

NASA Astrophysics Data System (ADS)

Kobayashi, Tsubasa; Cha, Yu-Rok; Kaji, Yuichi; Oshika, Tetsuro; Leproux, Philippe; Couderc, Vincent; Kano, Hideaki

2018-02-01

Acanthamoeba keratitis is a disease in which amoebae named Acanthamoeba invade the cornea of an eye. To diagnose this disease before it becomes serious, it is important to detect the cyst state of Acanthamoeba in the early stage of infection. In the present study, we explored spectroscopic signitures of the cyst state of Acanthamoeba using multimodal nonlinear optical microscopy with the channels of multiplex coherent anti-Stokes Raman scattering (CARS), second harmonic generation (SHG), and third harmonic generation (THG). A sharp band at around 1603 cm-1 in the CARS (Im[χ(3)]) spectrum was found at the cyst state of Acanthamoeba, which possibly originates from ergosterol and/or 7-dehydrostigmasterol. It can be used as a maker band of Acanthamoeba for medical treatment. Keyword: Acanthamoeba keratitis, coherent anti-Stokes Raman scattering, CARS, second harmonic generation, SHG, microspectroscopy, multiphoton microscopy

Targeted delivery of cancer-specific multimodal contrast agents for intraoperative detection of tumor boundaries and therapeutic margins

NASA Astrophysics Data System (ADS)

Xu, Ronald X.; Xu, Jeff S.; Huang, Jiwei; Tweedle, Michael F.; Schmidt, Carl; Povoski, Stephen P.; Martin, Edward W.

2010-02-01

Background: Accurate assessment of tumor boundaries and intraoperative detection of therapeutic margins are important oncologic principles for minimal recurrence rates and improved long-term outcomes. However, many existing cancer imaging tools are based on preoperative image acquisition and do not provide real-time intraoperative information that supports critical decision-making in the operating room. Method: Poly lactic-co-glycolic acid (PLGA) microbubbles (MBs) and nanobubbles (NBs) were synthesized by a modified double emulsion method. The MB and NB surfaces were conjugated with CC49 antibody to target TAG-72 antigen, a human glycoprotein complex expressed in many epithelial-derived cancers. Multiple imaging agents were encapsulated in MBs and NBs for multimodal imaging. Both one-step and multi-step cancer targeting strategies were explored. Active MBs/NBs were also fabricated for therapeutic margin assessment in cancer ablation therapies. Results: The multimodal contrast agents and the cancer-targeting strategies were tested on tissue simulating phantoms, LS174 colon cancer cell cultures, and cancer xenograft nude mice. Concurrent multimodal imaging was demonstrated using fluorescence and ultrasound imaging modalities. Technical feasibility of using active MBs and portable imaging tools such as ultrasound for intraoperative therapeutic margin assessment was demonstrated in a biological tissue model. Conclusion: The cancer-specific multimodal contrast agents described in this paper have the potential for intraoperative detection of tumor boundaries and therapeutic margins.

Carbon Tube Electrodes for Electrocardiography-Gated Cardiac Multimodality Imaging in Mice

PubMed Central

Choquet, Philippe; Goetz, Christian; Aubertin, Gaelle; Hubele, Fabrice; Sannié, Sébastien; Constantinesco, André

2011-01-01

This report describes a simple design of noninvasive carbon tube electrodes that facilitates electrocardiography (ECG) in mice during cardiac multimodality preclinical imaging. Both forepaws and the left hindpaw, covered by conductive gel, of mice were placed into the openings of small carbon tubes. Cardiac ECG-gated single-photon emission CT, X-ray CT, and MRI were tested (n = 60) in 20 mice. For all applications, electrodes were used in a warmed multimodality imaging cell. A heart rate of 563 ± 48 bpm was recorded from anesthetized mice regardless of the imaging technique used, with acquisition times ranging from 1 to 2 h. PMID:21333165

Development of a precision multimodal surgical navigation system for lung robotic segmentectomy

PubMed Central

Soldea, Valentin; Lachkar, Samy; Rinieri, Philippe; Sarsam, Mathieu; Bottet, Benjamin; Peillon, Christophe

2018-01-01

Minimally invasive sublobar anatomical resection is becoming more and more popular to manage early lung lesions. Robotic-assisted thoracic surgery (RATS) is unique in comparison with other minimally invasive techniques. Indeed, RATS is able to better integrate multiple streams of information including advanced imaging techniques, in an immersive experience at the level of the robotic console. Our aim was to describe three-dimensional (3D) imaging throughout the surgical procedure from preoperative planning to intraoperative assistance and complementary investigations such as radial endobronchial ultrasound (R-EBUS) and virtual bronchoscopy for pleural dye marking. All cases were operated using the DaVinci SystemTM. Modelisation was provided by Visible Patient™ (Strasbourg, France). Image integration in the operative field was achieved using the Tile Pro multi display input of the DaVinci console. Our experience was based on 114 robotic segmentectomies performed between January 2012 and October 2017. The clinical value of 3D imaging integration was evaluated in 2014 in a pilot study. Progressively, we have reached the conclusion that the use of such an anatomic model improves the safety and reliability of procedures. The multimodal system including 3D imaging has been used in more than 40 patients so far and demonstrated a perfect operative anatomic accuracy. Currently, we are developing an original virtual reality experience by exploring 3D imaging models at the robotic console level. The act of operating is being transformed and the surgeon now oversees a complex system that improves decision making. PMID:29785294

Development of a precision multimodal surgical navigation system for lung robotic segmentectomy.

PubMed

Baste, Jean Marc; Soldea, Valentin; Lachkar, Samy; Rinieri, Philippe; Sarsam, Mathieu; Bottet, Benjamin; Peillon, Christophe

2018-04-01

Minimally invasive sublobar anatomical resection is becoming more and more popular to manage early lung lesions. Robotic-assisted thoracic surgery (RATS) is unique in comparison with other minimally invasive techniques. Indeed, RATS is able to better integrate multiple streams of information including advanced imaging techniques, in an immersive experience at the level of the robotic console. Our aim was to describe three-dimensional (3D) imaging throughout the surgical procedure from preoperative planning to intraoperative assistance and complementary investigations such as radial endobronchial ultrasound (R-EBUS) and virtual bronchoscopy for pleural dye marking. All cases were operated using the DaVinci System TM . Modelisation was provided by Visible Patient™ (Strasbourg, France). Image integration in the operative field was achieved using the Tile Pro multi display input of the DaVinci console. Our experience was based on 114 robotic segmentectomies performed between January 2012 and October 2017. The clinical value of 3D imaging integration was evaluated in 2014 in a pilot study. Progressively, we have reached the conclusion that the use of such an anatomic model improves the safety and reliability of procedures. The multimodal system including 3D imaging has been used in more than 40 patients so far and demonstrated a perfect operative anatomic accuracy. Currently, we are developing an original virtual reality experience by exploring 3D imaging models at the robotic console level. The act of operating is being transformed and the surgeon now oversees a complex system that improves decision making.

Volume curtaining: a focus+context effect for multimodal volume visualization

NASA Astrophysics Data System (ADS)

Fairfield, Adam J.; Plasencia, Jonathan; Jang, Yun; Theodore, Nicholas; Crawford, Neil R.; Frakes, David H.; Maciejewski, Ross

2014-03-01

In surgical preparation, physicians will often utilize multimodal imaging scans to capture complementary information to improve diagnosis and to drive patient-specific treatment. These imaging scans may consist of data from magnetic resonance imaging (MR), computed tomography (CT), or other various sources. The challenge in using these different modalities is that the physician must mentally map the two modalities together during the diagnosis and planning phase. Furthermore, the different imaging modalities will be generated at various resolutions as well as slightly different orientations due to patient placement during scans. In this work, we present an interactive system for multimodal data fusion, analysis and visualization. Developed with partners from neurological clinics, this work discusses initial system requirements and physician feedback at the various stages of component development. Finally, we present a novel focus+context technique for the interactive exploration of coregistered multi-modal data.

Quantitative multimodality imaging in cancer research and therapy.

PubMed

Yankeelov, Thomas E; Abramson, Richard G; Quarles, C Chad

2014-11-01

Advances in hardware and software have enabled the realization of clinically feasible, quantitative multimodality imaging of tissue pathophysiology. Earlier efforts relating to multimodality imaging of cancer have focused on the integration of anatomical and functional characteristics, such as PET-CT and single-photon emission CT (SPECT-CT), whereas more-recent advances and applications have involved the integration of multiple quantitative, functional measurements (for example, multiple PET tracers, varied MRI contrast mechanisms, and PET-MRI), thereby providing a more-comprehensive characterization of the tumour phenotype. The enormous amount of complementary quantitative data generated by such studies is beginning to offer unique insights into opportunities to optimize care for individual patients. Although important technical optimization and improved biological interpretation of multimodality imaging findings are needed, this approach can already be applied informatively in clinical trials of cancer therapeutics using existing tools. These concepts are discussed herein.

Multimode intravascular RF coil for MRI-guided interventions.

PubMed

Kurpad, Krishna N; Unal, Orhan

2011-04-01

To demonstrate the feasibility of using a single intravascular radiofrequency (RF) probe connected to the external magnetic resonance imaging (MRI) system via a single coaxial cable to perform active tip tracking and catheter visualization and high signal-to-noise ratio (SNR) intravascular imaging. A multimode intravascular RF coil was constructed on a 6F balloon catheter and interfaced to a 1.5T MRI scanner via a decoupling circuit. Bench measurements of coil impedances were followed by imaging experiments in saline and phantoms. The multimode coil behaves as an inductively coupled transmit coil. The forward-looking capability of 6 mm was measured. A greater than 3-fold increase in SNR compared to conventional imaging using optimized external coil was demonstrated. Simultaneous active tip tracking and catheter visualization was demonstrated. It is feasible to perform 1) active tip tracking, 2) catheter visualization, and 3) high SNR imaging using a single multimode intravascular RF coil that is connected to the external system via a single coaxial cable. Copyright © 2011 Wiley-Liss, Inc.

Log-Gabor Energy Based Multimodal Medical Image Fusion in NSCT Domain

PubMed Central

Yang, Yong; Tong, Song; Huang, Shuying; Lin, Pan

2014-01-01

Multimodal medical image fusion is a powerful tool in clinical applications such as noninvasive diagnosis, image-guided radiotherapy, and treatment planning. In this paper, a novel nonsubsampled Contourlet transform (NSCT) based method for multimodal medical image fusion is presented, which is approximately shift invariant and can effectively suppress the pseudo-Gibbs phenomena. The source medical images are initially transformed by NSCT followed by fusing low- and high-frequency components. The phase congruency that can provide a contrast and brightness-invariant representation is applied to fuse low-frequency coefficients, whereas the Log-Gabor energy that can efficiently determine the frequency coefficients from the clear and detail parts is employed to fuse the high-frequency coefficients. The proposed fusion method has been compared with the discrete wavelet transform (DWT), the fast discrete curvelet transform (FDCT), and the dual tree complex wavelet transform (DTCWT) based image fusion methods and other NSCT-based methods. Visually and quantitatively experimental results indicate that the proposed fusion method can obtain more effective and accurate fusion results of multimodal medical images than other algorithms. Further, the applicability of the proposed method has been testified by carrying out a clinical example on a woman affected with recurrent tumor images. PMID:25214889

Multi-mode of Four and Six Wave Parametric Amplified Process

NASA Astrophysics Data System (ADS)

Zhu, Dayu; Yang, Yiheng; Zhang, Da; Liu, Ruizhou; Ma, Danmeng; Li, Changbiao; Zhang, Yanpeng

2017-03-01

Multiple quantum modes in correlated fields are essential for future quantum information processing and quantum computing. Here we report the generation of multi-mode phenomenon through parametric amplified four- and six-wave mixing processes in a rubidium atomic ensemble. The multi-mode properties in both frequency and spatial domains are studied. On one hand, the multi-mode behavior is dominantly controlled by the intensity of external dressing effect, or nonlinear phase shift through internal dressing effect, in frequency domain; on the other hand, the multi-mode behavior is visually demonstrated from the images of the biphoton fields directly, in spatial domain. Besides, the correlation of the two output fields is also demonstrated in both domains. Our approach supports efficient applications for scalable quantum correlated imaging.

Multi-mode of Four and Six Wave Parametric Amplified Process.

PubMed

Zhu, Dayu; Yang, Yiheng; Zhang, Da; Liu, Ruizhou; Ma, Danmeng; Li, Changbiao; Zhang, Yanpeng

2017-03-03

Multiple quantum modes in correlated fields are essential for future quantum information processing and quantum computing. Here we report the generation of multi-mode phenomenon through parametric amplified four- and six-wave mixing processes in a rubidium atomic ensemble. The multi-mode properties in both frequency and spatial domains are studied. On one hand, the multi-mode behavior is dominantly controlled by the intensity of external dressing effect, or nonlinear phase shift through internal dressing effect, in frequency domain; on the other hand, the multi-mode behavior is visually demonstrated from the images of the biphoton fields directly, in spatial domain. Besides, the correlation of the two output fields is also demonstrated in both domains. Our approach supports efficient applications for scalable quantum correlated imaging.

Multimodal optoacoustic and multiphoton fluorescence microscopy

NASA Astrophysics Data System (ADS)

Sela, Gali; Razansky, Daniel; Shoham, Shy

2013-03-01

Multiphoton microscopy is a powerful imaging modality that enables structural and functional imaging with cellular and sub-cellular resolution, deep within biological tissues. Yet, its main contrast mechanism relies on extrinsically administered fluorescent indicators. Here we developed a system for simultaneous multimodal optoacoustic and multiphoton fluorescence 3D imaging, which attains both absorption and fluorescence-based contrast by integrating an ultrasonic transducer into a two-photon laser scanning microscope. The system is readily shown to enable acquisition of multimodal microscopic images of fluorescently labeled targets and cell cultures as well as intrinsic absorption-based images of pigmented biological tissue. During initial experiments, it was further observed that that detected optoacoustically-induced response contains low frequency signal variations, presumably due to cavitation-mediated signal generation by the high repetition rate (80MHz) near IR femtosecond laser. The multimodal system may provide complementary structural and functional information to the fluorescently labeled tissue, by superimposing optoacoustic images of intrinsic tissue chromophores, such as melanin deposits, pigmentation, and hemoglobin or other extrinsic particle or dye-based markers highly absorptive in the NIR spectrum.

Recommendations on nuclear and multimodality imaging in IE and CIED infections.

PubMed

Erba, Paola Anna; Lancellotti, Patrizio; Vilacosta, Isidre; Gaemperli, Oliver; Rouzet, Francois; Hacker, Marcus; Signore, Alberto; Slart, Riemer H J A; Habib, Gilbert

2018-05-24

In the latest update of the European Society of Cardiology (ESC) guidelines for the management of infective endocarditis (IE), imaging is positioned at the centre of the diagnostic work-up so that an early and accurate diagnosis can be reached. Besides echocardiography, contrast-enhanced CT (ce-CT), radiolabelled leucocyte (white blood cell, WBC) SPECT/CT and [ 18 F]FDG PET/CT are included as diagnostic tools in the diagnostic flow chart for IE. Following the clinical guidelines that provided a straightforward message on the role of multimodality imaging, we believe that it is highly relevant to produce specific recommendations on nuclear multimodality imaging in IE and cardiac implantable electronic device infections. In these procedural recommendations we therefore describe in detail the technical and practical aspects of WBC SPECT/CT and [ 18 F]FDG PET/CT, including ce-CT acquisition protocols. We also discuss the advantages and limitations of each procedure, specific pitfalls when interpreting images, and the most important results from the literature, and also provide recommendations on the appropriate use of multimodality imaging.

Multimodal swept-source spectrally encoded scanning laser ophthalmoscopy and optical coherence tomography at 400 kHz

NASA Astrophysics Data System (ADS)

El-Haddad, Mohamed T.; Joos, Karen M.; Patel, Shriji N.; Tao, Yuankai K.

2017-02-01

Multimodal imaging systems that combine scanning laser ophthalmoscopy (SLO) and optical coherence tomography (OCT) have demonstrated the utility of concurrent en face and volumetric imaging for aiming, eye tracking, bulk motion compensation, mosaicking, and contrast enhancement. However, this additional functionality trades off with increased system complexity and cost because both SLO and OCT generally require dedicated light sources, galvanometer scanners, relay and imaging optics, detectors, and control and digitization electronics. We previously demonstrated multimodal ophthalmic imaging using swept-source spectrally encoded SLO and OCT (SS-SESLO-OCT). Here, we present system enhancements and a new optical design that increase our SS-SESLO-OCT data throughput by >7x and field-of-view (FOV) by >4x. A 200 kHz 1060 nm Axsun swept-source was optically buffered to 400 kHz sweep-rate, and SESLO and OCT were simultaneously digitized on dual input channels of a 4 GS/s digitizer at 1.2 GS/s per channel using a custom k-clock. We show in vivo human imaging of the anterior segment out to the limbus and retinal fundus over a >40° FOV. In addition, nine overlapping volumetric SS-SESLO-OCT volumes were acquired under video-rate SESLO preview and guidance. In post-processing, all nine SESLO images and en face projections of the corresponding OCT volumes were mosaicked to show widefield multimodal fundus imaging with a >80° FOV. Concurrent multimodal SS-SESLO-OCT may have applications in clinical diagnostic imaging by enabling aiming, image registration, and multi-field mosaicking and benefit intraoperative imaging by allowing for real-time surgical feedback, instrument tracking, and overlays of computationally extracted image-based surrogate biomarkers of disease.

Cross-modal learning to rank via latent joint representation.

PubMed

Wu, Fei; Jiang, Xinyang; Li, Xi; Tang, Siliang; Lu, Weiming; Zhang, Zhongfei; Zhuang, Yueting

2015-05-01

Cross-modal ranking is a research topic that is imperative to many applications involving multimodal data. Discovering a joint representation for multimodal data and learning a ranking function are essential in order to boost the cross-media retrieval (i.e., image-query-text or text-query-image). In this paper, we propose an approach to discover the latent joint representation of pairs of multimodal data (e.g., pairs of an image query and a text document) via a conditional random field and structural learning in a listwise ranking manner. We call this approach cross-modal learning to rank via latent joint representation (CML²R). In CML²R, the correlations between multimodal data are captured in terms of their sharing hidden variables (e.g., topics), and a hidden-topic-driven discriminative ranking function is learned in a listwise ranking manner. The experiments show that the proposed approach achieves a good performance in cross-media retrieval and meanwhile has the capability to learn the discriminative representation of multimodal data.

Multi-Modality Cascaded Convolutional Neural Networks for Alzheimer's Disease Diagnosis.

PubMed

Liu, Manhua; Cheng, Danni; Wang, Kundong; Wang, Yaping

2018-03-23

Accurate and early diagnosis of Alzheimer's disease (AD) plays important role for patient care and development of future treatment. Structural and functional neuroimages, such as magnetic resonance images (MRI) and positron emission tomography (PET), are providing powerful imaging modalities to help understand the anatomical and functional neural changes related to AD. In recent years, machine learning methods have been widely studied on analysis of multi-modality neuroimages for quantitative evaluation and computer-aided-diagnosis (CAD) of AD. Most existing methods extract the hand-craft imaging features after image preprocessing such as registration and segmentation, and then train a classifier to distinguish AD subjects from other groups. This paper proposes to construct cascaded convolutional neural networks (CNNs) to learn the multi-level and multimodal features of MRI and PET brain images for AD classification. First, multiple deep 3D-CNNs are constructed on different local image patches to transform the local brain image into more compact high-level features. Then, an upper high-level 2D-CNN followed by softmax layer is cascaded to ensemble the high-level features learned from the multi-modality and generate the latent multimodal correlation features of the corresponding image patches for classification task. Finally, these learned features are combined by a fully connected layer followed by softmax layer for AD classification. The proposed method can automatically learn the generic multi-level and multimodal features from multiple imaging modalities for classification, which are robust to the scale and rotation variations to some extent. No image segmentation and rigid registration are required in pre-processing the brain images. Our method is evaluated on the baseline MRI and PET images of 397 subjects including 93 AD patients, 204 mild cognitive impairment (MCI, 76 pMCI +128 sMCI) and 100 normal controls (NC) from Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Experimental results show that the proposed method achieves an accuracy of 93.26% for classification of AD vs. NC and 82.95% for classification pMCI vs. NC, demonstrating the promising classification performance.

Crucial breakthrough of second near-infrared biological window fluorophores: design and synthesis toward multimodal imaging and theranostics

DOE PAGES

He, Shuqing; Song, Jun; Qu, Junle; ...

2018-01-01

Recent advances in the chemical design and synthesis of fluorophores in the second near-infrared biological window (NIR-II) for multimodal imaging and theranostics are summarized and highlighted in this review article.

Crucial breakthrough of second near-infrared biological window fluorophores: design and synthesis toward multimodal imaging and theranostics

DOE Office of Scientific and Technical Information (OSTI.GOV)

He, Shuqing; Song, Jun; Qu, Junle

Recent advances in the chemical design and synthesis of fluorophores in the second near-infrared biological window (NIR-II) for multimodal imaging and theranostics are summarized and highlighted in this review article.

The sweet spot: FDG and other 2-carbon glucose analogs for multi-modal metabolic imaging of tumor metabolism

PubMed Central

Cox, Benjamin L; Mackie, Thomas R; Eliceiri, Kevin W

2015-01-01

Multi-modal imaging approaches of tumor metabolism that provide improved specificity, physiological relevance and spatial resolution would improve diagnosing of tumors and evaluation of tumor progression. Currently, the molecular probe FDG, glucose fluorinated with 18F at the 2-carbon, is the primary metabolic approach for clinical diagnostics with PET imaging. However, PET lacks the resolution necessary to yield intratumoral distributions of deoxyglucose, on the cellular level. Multi-modal imaging could elucidate this problem, but requires the development of new glucose analogs that are better suited for other imaging modalities. Several such analogs have been created and are reviewed here. Also reviewed are several multi-modal imaging studies that have been performed that attempt to shed light on the cellular distribution of glucose analogs within tumors. Some of these studies are performed in vitro, while others are performed in vivo, in an animal model. The results from these studies introduce a visualization gap between the in vitro and in vivo studies that, if solved, could enable the early detection of tumors, the high resolution monitoring of tumors during treatment, and the greater accuracy in assessment of different imaging agents. PMID:25625022

Towards an ultra-thin medical endoscope: multimode fibre as a wide-field image transferring medium

NASA Astrophysics Data System (ADS)

Duriš, Miroslav; Bradu, Adrian; Podoleanu, Adrian; Hughes, Michael

2018-03-01

Multimode optical fibres are attractive for biomedical and industrial applications such as endoscopes because of the small cross section and imaging resolution they can provide in comparison to widely-used fibre bundles. However, the image is randomly scrambled by propagation through a multimode fibre. Even though the scrambling is unpredictable, it is deterministic, and therefore the scrambling can be reversed. To unscramble the image, we treat the multimode fibre as a linear, disordered scattering medium. To calibrate, we scan a focused beam of coherent light over thousands of different beam positions at the distal end and record complex fields at the proximal end of the fibre. This way, the inputoutput response of the system is determined, which then allows computational reconstruction of reflection-mode images. However, there remains the problem of illuminating the tissue via the fibre while avoiding back reflections from the proximal face. To avoid this drawback, we provide here the first preliminary confirmation that an image can be transferred through a 2x2 fibre coupler, with the sample at its distal port interrogated in reflection. Light is injected into one port for illumination and then collected from a second port for imaging.

The new frontiers of multimodality and multi-isotope imaging

NASA Astrophysics Data System (ADS)

Behnam Azad, Babak; Nimmagadda, Sridhar

2014-06-01

Technological advances in imaging systems and the development of target specific imaging tracers has been rapidly growing over the past two decades. Recent progress in "all-in-one" imaging systems that allow for automated image coregistration has significantly added to the growth of this field. These developments include ultra high resolution PET and SPECT scanners that can be integrated with CT or MR resulting in PET/CT, SPECT/CT, SPECT/PET and PET/MRI scanners for simultaneous high resolution high sensitivity anatomical and functional imaging. These technological developments have also resulted in drastic enhancements in image quality and acquisition time while eliminating cross compatibility issues between modalities. Furthermore, the most cutting edge technology, though mostly preclinical, also allows for simultaneous multimodality multi-isotope image acquisition and image reconstruction based on radioisotope decay characteristics. These scientific advances, in conjunction with the explosion in the development of highly specific multimodality molecular imaging agents, may aid in realizing simultaneous imaging of multiple biological processes and pave the way towards more efficient diagnosis and improved patient care.

Multimodal imaging of lung cancer and its microenvironment (Conference Presentation)

NASA Astrophysics Data System (ADS)

Hariri, Lida P.; Niederst, Matthew J.; Mulvey, Hillary; Adams, David C.; Hu, Haichuan; Chico Calero, Isabel; Szabari, Margit V.; Vakoc, Benjamin J.; Hasan, Tayyaba; Bouma, Brett E.; Engelman, Jeffrey A.; Suter, Melissa J.

2016-03-01

Despite significant advances in targeted therapies for lung cancer, nearly all patients develop drug resistance within 6-12 months and prognosis remains poor. Developing drug resistance is a progressive process that involves tumor cells and their microenvironment. We hypothesize that microenvironment factors alter tumor growth and response to targeted therapy. We conducted in vitro studies in human EGFR-mutant lung carcinoma cells, and demonstrated that factors secreted from lung fibroblasts results in increased tumor cell survival during targeted therapy with EGFR inhibitor, gefitinib. We also demonstrated that increased environment stiffness results in increased tumor survival during gefitinib therapy. In order to test our hypothesis in vivo, we developed a multimodal optical imaging protocol for preclinical intravital imaging in mouse models to assess tumor and its microenvironment over time. We have successfully conducted multimodal imaging of dorsal skinfold chamber (DSC) window mice implanted with GFP-labeled human EGFR mutant lung carcinoma cells and visualized changes in tumor development and microenvironment facets over time. Multimodal imaging included structural OCT to assess tumor viability and necrosis, polarization-sensitive OCT to measure tissue birefringence for collagen/fibroblast detection, and Doppler OCT to assess tumor vasculature. Confocal imaging was also performed for high-resolution visualization of EGFR-mutant lung cancer cells labeled with GFP, and was coregistered with OCT. Our results demonstrated that stromal support and vascular growth are essential to tumor progression. Multimodal imaging is a useful tool to assess tumor and its microenvironment over time.

Multimodal flexible cystoscopy for creating co-registered panoramas of the bladder urothelium

NASA Astrophysics Data System (ADS)

Seibel, Eric J.; Soper, Timothy D.; Burkhardt, Matthew R.; Porter, Michael P.; Yoon, W. Jong

2012-02-01

Bladder cancer is the most expensive cancer to treat due to the high rate of recurrence. Though white light cystoscopy is the gold standard for bladder cancer surveillance, the advent of fluorescence biomarkers provides an opportunity to improve sensitivity for early detection and reduced recurrence resulting from more accurate excision. Ideally, fluorescence information could be combined with standard reflectance images to provide multimodal views of the bladder wall. The scanning fiber endoscope (SFE) of 1.2mm in diameter is able to acquire wide-field multimodal video from a bladder phantom with fluorescence cancer "hot-spots". The SFE generates images by scanning red, green, and blue (RGB) laser light and detects the backscatter signal for reflectance video of 500-line resolution at 30 frames per second. We imaged a bladder phantom with painted vessels and mimicked fluorescent lesions by applying green fluorescent microspheres to the surface. By eliminating the green laser illumination, simultaneous reflectance and fluorescence images can be acquired at the same field of view, resolution, and frame rate. Moreover, the multimodal SFE is combined with a robotic steering mechanism and image stitching software as part of a fully automated bladder surveillance system. Using this system, the SFE can be reliably articulated over the entire 360° bladder surface. Acquired images can then be stitched into a multimodal 3D panorama of the bladder using software developed in our laboratory. In each panorama, the fluorescence images are exactly co-registered with RGB reflectance.

Development of computational small animal models and their applications in preclinical imaging and therapy research

DOE Office of Scientific and Technical Information (OSTI.GOV)

Xie, Tianwu; Zaidi, Habib, E-mail: habib.zaidi@hcuge.ch; Geneva Neuroscience Center, Geneva University, Geneva CH-1205

The development of multimodality preclinical imaging techniques and the rapid growth of realistic computer simulation tools have promoted the construction and application of computational laboratory animal models in preclinical research. Since the early 1990s, over 120 realistic computational animal models have been reported in the literature and used as surrogates to characterize the anatomy of actual animals for the simulation of preclinical studies involving the use of bioluminescence tomography, fluorescence molecular tomography, positron emission tomography, single-photon emission computed tomography, microcomputed tomography, magnetic resonance imaging, and optical imaging. Other applications include electromagnetic field simulation, ionizing and nonionizing radiation dosimetry, and themore » development and evaluation of new methodologies for multimodality image coregistration, segmentation, and reconstruction of small animal images. This paper provides a comprehensive review of the history and fundamental technologies used for the development of computational small animal models with a particular focus on their application in preclinical imaging as well as nonionizing and ionizing radiation dosimetry calculations. An overview of the overall process involved in the design of these models, including the fundamental elements used for the construction of different types of computational models, the identification of original anatomical data, the simulation tools used for solving various computational problems, and the applications of computational animal models in preclinical research. The authors also analyze the characteristics of categories of computational models (stylized, voxel-based, and boundary representation) and discuss the technical challenges faced at the present time as well as research needs in the future.« less

New Technologies, New Possibilities for the Arts and Multimodality in English Language Arts

ERIC Educational Resources Information Center

Williams, Wendy R.

2014-01-01

This article discusses the arts, multimodality, and new technologies in English language arts. It then turns to the example of the illuminated text--a multimodal book report consisting of animated text, music, and images--to consider how art, multimodality, and technology can work together to support students' reading of literature and inspire…

Combining kriging, multispectral and multimodal microscopy to resolve malaria-infected erythrocyte contents.

PubMed

Dabo-Niang, S; Zoueu, J T

2012-09-01

In this communication, we demonstrate how kriging, combine with multispectral and multimodal microscopy can enhance the resolution of malaria-infected images and provide more details on their composition, for analysis and diagnosis. The results of this interpolation applied to the two principal components of multispectral and multimodal images illustrate that the examination of the content of Plasmodium falciparum infected human erythrocyte is improved. © 2012 The Authors Journal of Microscopy © 2012 Royal Microscopical Society.

Multifocus confocal Raman microspectroscopy for fast multimode vibrational imaging of living cells.

PubMed

Okuno, Masanari; Hamaguchi, Hiro-o

2010-12-15

We have developed a multifocus confocal Raman microspectroscopic system for the fast multimode vibrational imaging of living cells. It consists of an inverted microscope equipped with a microlens array, a pinhole array, a fiber bundle, and a multichannel Raman spectrometer. Forty-eight Raman spectra from 48 foci under the microscope are simultaneously obtained by using multifocus excitation and image-compression techniques. The multifocus confocal configuration suppresses the background generated from the cover glass and the cell culturing medium so that high-contrast images are obtainable with a short accumulation time. The system enables us to obtain multimode (10 different vibrational modes) vibrational images of living cells in tens of seconds with only 1 mW laser power at one focal point. This image acquisition time is more than 10 times faster than that in conventional single-focus Raman microspectroscopy.

Bulimia nervosa symptoms in the multimodal treatment study of children with ADHD.

PubMed

Mikami, Amori Yee; Hinshaw, Stephen P; Arnold, L Eugene; Hoza, Betsy; Hechtman, Lily; Newcorn, Jeffrey H; Abikoff, Howard B

2010-04-01

We investigated body image dissatisfaction and bingeing/purging characteristics of bulimia nervosa (BN) in the ongoing prospective follow-up of the Multimodal Treatment Study of Children with attention-deficit/hyperactivity disorder (ADHD). Participants were 337 boys and 95 girls with ADHD and 211 boys and 53 girls forming a local normative comparison group (LNCG), reassessed in midadolescence (mean age, 16.4), 8 years after original recruitment. Youth with childhood ADHD showed more BN symptoms in midadolescence than did LNCG youth, and girls demonstrated more BN symptoms than did boys, with effect sizes between small and medium. Childhood impulsivity, as opposed to hyperactivity or inattention, best predicted adolescent BN symptoms, particularly for girls. Among youth with ADHD, treatment received during the follow-up period was not associated with BN pathology. Both boys and girls with ADHD may be at risk for BN symptoms in adolescence because of the impulsivity central to both disorders.

Integrated photoacoustic microscopy, optical coherence tomography, and fluorescence microscopy for multimodal chorioretinal imaging

NASA Astrophysics Data System (ADS)

Tian, Chao; Zhang, Wei; Nguyen, Van Phuc; Huang, Ziyi; Wang, Xueding; Paulus, Yannis M.

2018-02-01

Current clinical available retinal imaging techniques have limitations, including limited depth of penetration or requirement for the invasive injection of exogenous contrast agents. Here, we developed a novel multimodal imaging system for high-speed, high-resolution retinal imaging of larger animals, such as rabbits. The system integrates three state-of-the-art imaging modalities, including photoacoustic microscopy (PAM), optical coherence tomography (OCT), and fluorescence microscopy (FM). In vivo experimental results of rabbit eyes show that the PAM is able to visualize laser-induced retinal burns and distinguish individual eye blood vessels using a laser exposure dose of 80 nJ, which is well below the American National Standards Institute (ANSI) safety limit 160 nJ. The OCT can discern different retinal layers and visualize laser burns and choroidal detachments. The novel multi-modal imaging platform holds great promise in ophthalmic imaging.

Multi-Modal Nano-Probes for Radionuclide and 5-color Near Infrared Optical Lymphatic Imaging

PubMed Central

Kobayashi, Hisataka; Koyama, Yoshinori; Barrett, Tristan; Hama, Yukihiro; Regino, Celeste A. S.; Shin, In Soo; Jang, Beom-Su; Le, Nhat; Paik, Chang H.; Choyke, Peter L.; Urano, Yasuteru

2008-01-01

Current contrast agents generally have one function and can only be imaged in monochrome, therefore, the majority of imaging methods can only impart uniparametric information. A single nano-particle has the potential to be loaded with multiple payloads. Such multi-modality probes have the ability to be imaged by more than one imaging technique, which could compensate for the weakness or even combine the advantages of each individual modality. Furthermore, optical imaging using different optical probes enables us to achieve multi-color in vivo imaging, wherein multiple parameters can be read from a single image. To allow differentiation of multiple optical signals in vivo, each probe should have a close but different near infrared emission. To this end, we synthesized nano-probes with multi-modal and multi-color potential, which employed a polyamidoamine dendrimer platform linked to both radionuclides and optical probes, permitting dual-modality scintigraphic and 5-color near infrared optical lymphatic imaging using a multiple excitation spectrally-resolved fluorescence imaging technique. PMID:19079788

Multimodal nanoprobes for radionuclide and five-color near-infrared optical lymphatic imaging.

PubMed

Kobayashi, Hisataka; Koyama, Yoshinori; Barrett, Tristan; Hama, Yukihiro; Regino, Celeste A S; Shin, In Soo; Jang, Beom-Su; Le, Nhat; Paik, Chang H; Choyke, Peter L; Urano, Yasuteru

2007-11-01

Current contrast agents generally have one function and can only be imaged in monochrome; therefore, the majority of imaging methods can only impart uniparametric information. A single nanoparticle has the potential to be loaded with multiple payloads. Such multimodality probes have the ability to be imaged by more than one imaging technique, which could compensate for the weakness or even combine the advantages of each individual modality. Furthermore, optical imaging using different optical probes enables us to achieve multicolor in vivo imaging, wherein multiple parameters can be read from a single image. To allow differentiation of multiple optical signals in vivo, each probe should have a close but different near-infrared emission. To this end, we synthesized nanoprobes with multimodal and multicolor potential, which employed a polyamidoamine dendrimer platform linked to both radionuclides and optical probes, permitting dual-modality scintigraphic and five-color near-infrared optical lymphatic imaging using a multiple-excitation spectrally resolved fluorescence imaging technique.

Multimodal digital color imaging system for facial skin lesion analysis

NASA Astrophysics Data System (ADS)

Bae, Youngwoo; Lee, Youn-Heum; Jung, Byungjo

2008-02-01

In dermatology, various digital imaging modalities have been used as an important tool to quantitatively evaluate the treatment effect of skin lesions. Cross-polarization color image was used to evaluate skin chromophores (melanin and hemoglobin) information and parallel-polarization image to evaluate skin texture information. In addition, UV-A induced fluorescent image has been widely used to evaluate various skin conditions such as sebum, keratosis, sun damages, and vitiligo. In order to maximize the evaluation efficacy of various skin lesions, it is necessary to integrate various imaging modalities into an imaging system. In this study, we propose a multimodal digital color imaging system, which provides four different digital color images of standard color image, parallel and cross-polarization color image, and UV-A induced fluorescent color image. Herein, we describe the imaging system and present the examples of image analysis. By analyzing the color information and morphological features of facial skin lesions, we are able to comparably and simultaneously evaluate various skin lesions. In conclusion, we are sure that the multimodal color imaging system can be utilized as an important assistant tool in dermatology.

Stereotactic radiation treatment planning and follow-up studies involving fused multimodality imaging.

PubMed

Hamm, Klaus D; Surber, Gunnar; Schmücking, Michael; Wurm, Reinhard E; Aschenbach, Rene; Kleinert, Gabriele; Niesen, A; Baum, Richard P

2004-11-01

Innovative new software solutions may enable image fusion to produce the desired data superposition for precise target definition and follow-up studies in radiosurgery/stereotactic radiotherapy in patients with intracranial lesions. The aim is to integrate the anatomical and functional information completely into the radiation treatment planning and to achieve an exact comparison for follow-up examinations. Special conditions and advantages of BrainLAB's fully automatic image fusion system are evaluated and described for this purpose. In 458 patients, the radiation treatment planning and some follow-up studies were performed using an automatic image fusion technique involving the use of different imaging modalities. Each fusion was visually checked and corrected as necessary. The computerized tomography (CT) scans for radiation treatment planning (slice thickness 1.25 mm), as well as stereotactic angiography for arteriovenous malformations, were acquired using head fixation with stereotactic arc or, in the case of stereotactic radiotherapy, with a relocatable stereotactic mask. Different magnetic resonance (MR) imaging sequences (T1, T2, and fluid-attenuated inversion-recovery images) and positron emission tomography (PET) scans were obtained without head fixation. Fusion results and the effects on radiation treatment planning and follow-up studies were analyzed. The precision level of the results of the automatic fusion depended primarily on the image quality, especially the slice thickness and the field homogeneity when using MR images, as well as on patient movement during data acquisition. Fully automated image fusion of different MR, CT, and PET studies was performed for each patient. Only in a few cases was it necessary to correct the fusion manually after visual evaluation. These corrections were minor and did not materially affect treatment planning. High-quality fusion of thin slices of a region of interest with a complete head data set could be performed easily. The target volume for radiation treatment planning could be accurately delineated using multimodal information provided by CT, MR, angiography, and PET studies. The fusion of follow-up image data sets yielded results that could be successfully compared and quantitatively evaluated. Depending on the quality of the originally acquired image, automated image fusion can be a very valuable tool, allowing for fast (approximately 1-2 minute) and precise fusion of all relevant data sets. Fused multimodality imaging improves the target volume definition for radiation treatment planning. High-quality follow-up image data sets should be acquired for image fusion to provide exactly comparable slices and volumetric results that will contribute to quality contol.

Multimodal system for the planning and guidance of bronchoscopy

NASA Astrophysics Data System (ADS)

Higgins, William E.; Cheirsilp, Ronnarit; Zang, Xiaonan; Byrnes, Patrick

2015-03-01

Many technical innovations in multimodal radiologic imaging and bronchoscopy have emerged recently in the effort against lung cancer. Modern X-ray computed-tomography (CT) scanners provide three-dimensional (3D) high-resolution chest images, positron emission tomography (PET) scanners give complementary molecular imaging data, and new integrated PET/CT scanners combine the strengths of both modalities. State-of-the-art bronchoscopes permit minimally invasive tissue sampling, with vivid endobronchial video enabling navigation deep into the airway-tree periphery, while complementary endobronchial ultrasound (EBUS) reveals local views of anatomical structures outside the airways. In addition, image-guided intervention (IGI) systems have proven their utility for CT-based planning and guidance of bronchoscopy. Unfortunately, no IGI system exists that integrates all sources effectively through the complete lung-cancer staging work flow. This paper presents a prototype of a computer-based multimodal IGI system that strives to fill this need. The system combines a wide range of automatic and semi-automatic image-processing tools for multimodal data fusion and procedure planning. It also provides a flexible graphical user interface for follow-on guidance of bronchoscopy/EBUS. Human-study results demonstrate the system's potential.

Structured illumination multimodal 3D-resolved quantitative phase and fluorescence sub-diffraction microscopy

PubMed Central

Chowdhury, Shwetadwip; Eldridge, Will J.; Wax, Adam; Izatt, Joseph A.

2017-01-01

Sub-diffraction resolution imaging has played a pivotal role in biological research by visualizing key, but previously unresolvable, sub-cellular structures. Unfortunately, applications of far-field sub-diffraction resolution are currently divided between fluorescent and coherent-diffraction regimes, and a multimodal sub-diffraction technique that bridges this gap has not yet been demonstrated. Here we report that structured illumination (SI) allows multimodal sub-diffraction imaging of both coherent quantitative-phase (QP) and fluorescence. Due to SI’s conventionally fluorescent applications, we first demonstrate the principle of SI-enabled three-dimensional (3D) QP sub-diffraction imaging with calibration microspheres. Image analysis confirmed enhanced lateral and axial resolutions over diffraction-limited QP imaging, and established striking parallels between coherent SI and conventional optical diffraction tomography. We next introduce an optical system utilizing SI to achieve 3D sub-diffraction, multimodal QP/fluorescent visualization of A549 biological cells fluorescently tagged for F-actin. Our results suggest that SI has a unique utility in studying biological phenomena with significant molecular, biophysical, and biochemical components. PMID:28663887

A Multimodal Search Engine for Medical Imaging Studies.

PubMed

Pinho, Eduardo; Godinho, Tiago; Valente, Frederico; Costa, Carlos

2017-02-01

The use of digital medical imaging systems in healthcare institutions has increased significantly, and the large amounts of data in these systems have led to the conception of powerful support tools: recent studies on content-based image retrieval (CBIR) and multimodal information retrieval in the field hold great potential in decision support, as well as for addressing multiple challenges in healthcare systems, such as computer-aided diagnosis (CAD). However, the subject is still under heavy research, and very few solutions have become part of Picture Archiving and Communication Systems (PACS) in hospitals and clinics. This paper proposes an extensible platform for multimodal medical image retrieval, integrated in an open-source PACS software with profile-based CBIR capabilities. In this article, we detail a technical approach to the problem by describing its main architecture and each sub-component, as well as the available web interfaces and the multimodal query techniques applied. Finally, we assess our implementation of the engine with computational performance benchmarks.

Multi-mode Intravascular RF Coil for MRI-guided Interventions

PubMed Central

Kurpad, Krishna N.; Unal, Orhan

2011-01-01

Purpose To demonstrate the feasibility of using a single intravascular RF probe connected to the external MRI system via a single coaxial cable to perform active tip tracking and catheter visualization, and high SNR intravascular imaging. Materials and Methods A multi-mode intravascular RF coil was constructed on a 6F balloon catheter and interfaced to a 1.5T MRI scanner via a decoupling circuit. Bench measurements of coil impedances were followed by imaging experiments in saline and phantoms. Results The multi-mode coil behaves as an inductively-coupled transmit coil. Forward looking capability of 6mm is measured. Greater than 3-fold increase in SNR compared to conventional imaging using optimized external coil is demonstrated. Simultaneous active tip tracking and catheter visualization is demonstrated. Conclusions It is feasible to perform 1) active tip tracking, 2) catheter visualization, and 3) high SNR imaging using a single multi-mode intravascular RF coil that is connected to the external system via a single coaxial cable. PMID:21448969

Identifying Multimodal Intermediate Phenotypes between Genetic Risk Factors and Disease Status in Alzheimer’s Disease

PubMed Central

Hao, Xiaoke; Yao, Xiaohui; Yan, Jingwen; Risacher, Shannon L.; Saykin, Andrew J.; Zhang, Daoqiang; Shen, Li

2016-01-01

Neuroimaging genetics has attracted growing attention and interest, which is thought to be a powerful strategy to examine the influence of genetic variants (i.e., single nucleotide polymorphisms (SNPs)) on structures or functions of human brain. In recent studies, univariate or multivariate regression analysis methods are typically used to capture the effective associations between genetic variants and quantitative traits (QTs) such as brain imaging phenotypes. The identified imaging QTs, although associated with certain genetic markers, may not be all disease specific. A useful, but underexplored, scenario could be to discover only those QTs associated with both genetic markers and disease status for revealing the chain from genotype to phenotype to symptom. In addition, multimodal brain imaging phenotypes are extracted from different perspectives and imaging markers consistently showing up in multimodalities may provide more insights for mechanistic understanding of diseases (i.e., Alzheimer’s disease (AD)). In this work, we propose a general framework to exploit multi-modal brain imaging phenotypes as intermediate traits that bridge genetic risk factors and multi-class disease status. We applied our proposed method to explore the relation between the well-known AD risk SNP APOE rs429358 and three baseline brain imaging modalities (i.e., structural magnetic resonance imaging (MRI), fluorodeoxyglucose positron emission tomography (FDG-PET) and F-18 florbetapir PET scans amyloid imaging (AV45)) from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. The empirical results demonstrate that our proposed method not only helps improve the performances of imaging genetic associations, but also discovers robust and consistent regions of interests (ROIs) across multi-modalities to guide the disease-induced interpretation. PMID:27277494

Patients at Risk for Peritoneal Surface Malignancy of Colorectal Cancer Origin: The Role of Second Look Laparotomy

PubMed Central

Brücher, Björn LDM; Stojadinovic, Alexander; Bilchik, Anton J.; Protic, Mladjan; Daumer, Martin; Nissan, Aviram; Avital, Itzhak

2013-01-01

Peritoneal surface malignancy (PSM) is a frequent occurrence in the natural history of colorectal cancer (CRC). Although significant advances have been made in screening of CRC, similar progress has yet to be made in the early detection of PSM of colorectal cancer origin. The fact that advanced CRC can be confined to the peritoneal surface without distant dissemination forms the basis for aggressive multi-modality therapy consisting of cytoreductive surgery (CRS) plus hyperthermic intra-peritoneal chemotherapy (HIPEC), and neoadjuvant and/or adjuvant systemic therapy. Reported overall survival with complete CRS+HIPEC exceeds that of systemic therapy alone for the treatment of PSM from CRC, underscoring the advantage of this multi-modality therapeutic approach. Patients with limited peritoneal disease from CRC can undergo complete cytoreduction, which is associated with the best reported outcomes. As early or limited peritoneal carcinomatosis is undetectable by conventional imaging modalities, second look laparotomy is an important means to identify disease in high-risk patients at a stage most amenable to complete cytoreduction. This review focuses on the identification of patients at risk for PSM from CRC and discusses the role of second look laparotomy. PMID:23459716

Multimodality bonchoscopic imaging of tracheopathica osteochondroplastica

NASA Astrophysics Data System (ADS)

Colt, Henri; Murgu, Septimiu D.; Ahn, Yeh-Chan; Brenner, Matt

2009-05-01

Results of a commercial optical coherence tomography system used as part of a multimodality diagnostic bronchoscopy platform are presented for a 61-year-old patient with central airway obstruction from tracheopathica osteochondroplastica. Comparison to results of white-light bronchoscopy, histology, and endobronchial ultrasound examination are accompanied by a discussion of resolution, penetration depth, contrast, and field of view of these imaging modalities. White-light bronchoscopy revealed irregularly shaped, firm submucosal nodules along cartilaginous structures of the anterior and lateral walls of the trachea, sparing the muscular posterior membrane. Endobronchial ultrasound showed a hyperechoic density of 0.4 cm thickness. optical coherence tomography (OCT) was performed using a commercially available, compact time-domain OCT system (Niris System, Imalux Corp., Cleveland, Ohio) with a magnetically actuating probe (two-dimensional, front imaging, and inside actuation). Images showed epithelium, upper submucosa, and osseous submucosal nodule layers corresponding with histopathology. To our knowledge, this is the first time these commercially available systems are used as part of a multimodality bronchoscopy platform to study diagnostic imaging of a benign disease causing central airway obstruction. Further studies are needed to optimize these systems for pulmonary applications and to determine how new-generation imaging modalities will be integrated into a multimodality bronchoscopy platform.

[Research on non-rigid registration of multi-modal medical image based on Demons algorithm].

PubMed

Hao, Peibo; Chen, Zhen; Jiang, Shaofeng; Wang, Yang

2014-02-01

Non-rigid medical image registration is a popular subject in the research areas of the medical image and has an important clinical value. In this paper we put forward an improved algorithm of Demons, together with the conservation of gray model and local structure tensor conservation model, to construct a new energy function processing multi-modal registration problem. We then applied the L-BFGS algorithm to optimize the energy function and solve complex three-dimensional data optimization problem. And finally we used the multi-scale hierarchical refinement ideas to solve large deformation registration. The experimental results showed that the proposed algorithm for large de formation and multi-modal three-dimensional medical image registration had good effects.

Multimodality Imaging Approach towards Primary Aortic Sarcomas Arising after Endovascular Abdominal Aortic Aneurysm Repair: Case Series Report.

PubMed

Kamran, Mudassar; Fowler, Kathryn J; Mellnick, Vincent M; Sicard, Gregorio A; Narra, Vamsi R

2016-06-01

Primary aortic neoplasms are rare. Aortic sarcoma arising after endovascular aneurysm repair (EVAR) is a scarce subset of primary aortic malignancies, reports of which are infrequent in the published literature. The diagnosis of aortic sarcoma is challenging due to its non-specific clinical presentation, and the prognosis is poor due to delayed diagnosis, rapid proliferation, and propensity for metastasis. Post-EVAR, aortic sarcomas may mimic other more common aortic processes on surveillance imaging. Radiologists are rarely knowledgeable about this rare entity for which multimodality imaging and awareness are invaluable in early diagnosis. A series of three pathologically confirmed cases are presented to display the multimodality imaging features and clinical presentations of aortic sarcoma arising after EVAR.

A multimodal imaging platform with integrated simultaneous photoacoustic microscopy, optical coherence tomography, optical Doppler tomography and fluorescence microscopy

NASA Astrophysics Data System (ADS)

Dadkhah, Arash; Zhou, Jun; Yeasmin, Nusrat; Jiao, Shuliang

2018-02-01

Various optical imaging modalities with different optical contrast mechanisms have been developed over the past years. Although most of these imaging techniques are being used in many biomedical applications and researches, integration of these techniques will allow researchers to reach the full potential of these technologies. Nevertheless, combining different imaging techniques is always challenging due to the difference in optical and hardware requirements for different imaging systems. Here, we developed a multimodal optical imaging system with the capability of providing comprehensive structural, functional and molecular information of living tissue in micrometer scale. This imaging system integrates photoacoustic microscopy (PAM), optical coherence tomography (OCT), optical Doppler tomography (ODT) and fluorescence microscopy in one platform. Optical-resolution PAM (OR-PAM) provides absorption-based imaging of biological tissues. Spectral domain OCT is able to provide structural information based on the scattering property of biological sample with no need for exogenous contrast agents. In addition, ODT is a functional extension of OCT with the capability of measurement and visualization of blood flow based on the Doppler effect. Fluorescence microscopy allows to reveal molecular information of biological tissue using autofluoresce or exogenous fluorophores. In-vivo as well as ex-vivo imaging studies demonstrated the capability of our multimodal imaging system to provide comprehensive microscopic information on biological tissues. Integrating all the aforementioned imaging modalities for simultaneous multimodal imaging has promising potential for preclinical research and clinical practice in the near future.

Global image registration using a symmetric block-matching approach

PubMed Central

Modat, Marc; Cash, David M.; Daga, Pankaj; Winston, Gavin P.; Duncan, John S.; Ourselin, Sébastien

2014-01-01

Abstract. Most medical image registration algorithms suffer from a directionality bias that has been shown to largely impact subsequent analyses. Several approaches have been proposed in the literature to address this bias in the context of nonlinear registration, but little work has been done for global registration. We propose a symmetric approach based on a block-matching technique and least-trimmed square regression. The proposed method is suitable for multimodal registration and is robust to outliers in the input images. The symmetric framework is compared with the original asymmetric block-matching technique and is shown to outperform it in terms of accuracy and robustness. The methodology presented in this article has been made available to the community as part of the NiftyReg open-source package. PMID:26158035

Multimodal targeted high relaxivity thermosensitive liposome for in vivo imaging

NASA Astrophysics Data System (ADS)

Kuijten, Maayke M. P.; Hannah Degeling, M.; Chen, John W.; Wojtkiewicz, Gregory; Waterman, Peter; Weissleder, Ralph; Azzi, Jamil; Nicolay, Klaas; Tannous, Bakhos A.

2015-11-01

Liposomes are spherical, self-closed structures formed by lipid bilayers that can encapsulate drugs and/or imaging agents in their hydrophilic core or within their membrane moiety, making them suitable delivery vehicles. We have synthesized a new liposome containing gadolinium-DOTA lipid bilayer, as a targeting multimodal molecular imaging agent for magnetic resonance and optical imaging. We showed that this liposome has a much higher molar relaxivities r1 and r2 compared to a more conventional liposome containing gadolinium-DTPA-BSA lipid. By incorporating both gadolinium and rhodamine in the lipid bilayer as well as biotin on its surface, we used this agent for multimodal imaging and targeting of tumors through the strong biotin-streptavidin interaction. Since this new liposome is thermosensitive, it can be used for ultrasound-mediated drug delivery at specific sites, such as tumors, and can be guided by magnetic resonance imaging.

Image recovery from defocused 2D fluorescent images in multimodal digital holographic microscopy.

PubMed

Quan, Xiangyu; Matoba, Osamu; Awatsuji, Yasuhiro

2017-05-01

A technique of three-dimensional (3D) intensity retrieval from defocused, two-dimensional (2D) fluorescent images in the multimodal digital holographic microscopy (DHM) is proposed. In the multimodal DHM, 3D phase and 2D fluorescence distributions are obtained simultaneously by an integrated system of an off-axis DHM and a conventional epifluorescence microscopy, respectively. This gives us more information of the target; however, defocused fluorescent images are observed due to the short depth of field. In this Letter, we propose a method to recover the defocused images based on the phase compensation and backpropagation from the defocused plane to the focused plane using the distance information that is obtained from a 3D phase distribution. By applying Zernike polynomial phase correction, we brought back the fluorescence intensity to the focused imaging planes. The experimental demonstration using fluorescent beads is presented, and the expected applications are suggested.

Real-time dynamic display of registered 4D cardiac MR and ultrasound images using a GPU

NASA Astrophysics Data System (ADS)

Zhang, Q.; Huang, X.; Eagleson, R.; Guiraudon, G.; Peters, T. M.

2007-03-01

In minimally invasive image-guided surgical interventions, different imaging modalities, such as magnetic resonance imaging (MRI), computed tomography (CT), and real-time three-dimensional (3D) ultrasound (US), can provide complementary, multi-spectral image information. Multimodality dynamic image registration is a well-established approach that permits real-time diagnostic information to be enhanced by placing lower-quality real-time images within a high quality anatomical context. For the guidance of cardiac procedures, it would be valuable to register dynamic MRI or CT with intraoperative US. However, in practice, either the high computational cost prohibits such real-time visualization of volumetric multimodal images in a real-world medical environment, or else the resulting image quality is not satisfactory for accurate guidance during the intervention. Modern graphics processing units (GPUs) provide the programmability, parallelism and increased computational precision to begin to address this problem. In this work, we first outline our research on dynamic 3D cardiac MR and US image acquisition, real-time dual-modality registration and US tracking. Then we describe image processing and optimization techniques for 4D (3D + time) cardiac image real-time rendering. We also present our multimodality 4D medical image visualization engine, which directly runs on a GPU in real-time by exploiting the advantages of the graphics hardware. In addition, techniques such as multiple transfer functions for different imaging modalities, dynamic texture binding, advanced texture sampling and multimodality image compositing are employed to facilitate the real-time display and manipulation of the registered dual-modality dynamic 3D MR and US cardiac datasets.

Fast and Robust Registration of Multimodal Remote Sensing Images via Dense Orientated Gradient Feature

NASA Astrophysics Data System (ADS)

Ye, Y.

2017-09-01

This paper presents a fast and robust method for the registration of multimodal remote sensing data (e.g., optical, LiDAR, SAR and map). The proposed method is based on the hypothesis that structural similarity between images is preserved across different modalities. In the definition of the proposed method, we first develop a pixel-wise feature descriptor named Dense Orientated Gradient Histogram (DOGH), which can be computed effectively at every pixel and is robust to non-linear intensity differences between images. Then a fast similarity metric based on DOGH is built in frequency domain using the Fast Fourier Transform (FFT) technique. Finally, a template matching scheme is applied to detect tie points between images. Experimental results on different types of multimodal remote sensing images show that the proposed similarity metric has the superior matching performance and computational efficiency than the state-of-the-art methods. Moreover, based on the proposed similarity metric, we also design a fast and robust automatic registration system for multimodal images. This system has been evaluated using a pair of very large SAR and optical images (more than 20000 × 20000 pixels). Experimental results show that our system outperforms the two popular commercial software systems (i.e. ENVI and ERDAS) in both registration accuracy and computational efficiency.

Water-stable NaLuF4-based upconversion nanophosphors with long-term validity for multimodal lymphatic imaging.

PubMed

Zhou, Jing; Zhu, Xingjun; Chen, Min; Sun, Yun; Li, Fuyou

2012-09-01

Multimodal imaging is rapidly becoming an important tool for biomedical applications because it can compensate for the deficiencies of individual imaging modalities. Herein, multifunctional NaLuF(4)-based upconversion nanoparticles (Lu-UCNPs) were synthesized though a facile one-step microemulsion method under ambient condition. The doping of lanthanide ions (Gd(3+), Yb(3+) and Er(3+)/Tm(3+)) endows the Lu-UCNPs with high T(1)-enhancement, bright upconversion luminescence (UCL) emissions, and excellent X-ray absorption coefficient. Moreover, the as-prepared Lu-UCNPs are stable in water for more than six months, due to the protection of sodium glutamate and diethylene triamine pentacetate acid (DTPA) coordinating ligands on the surface. Lu-UCNPs have been successfully applied to the trimodal CT/MR/UCL lymphatic imaging on the modal of small animals. It is worth noting that Lu-UCNPs could be used for imaging even after preserving for over six months. In vitro transmission electron microscope (TEM), methyl thiazolyl tetrazolium (MTT) assay and histological analysis demonstrated that Lu-UCNPs exhibited low toxicity on living systems. Therefore, Lu-UCNPs could be multimodal agents for CT/MR/UCL imaging, and the concept can be served as a platform technology for the next-generation of probes for multimodal imaging. Copyright © 2012 Elsevier Ltd. All rights reserved.

ADMultiImg: a novel missing modality transfer learning based CAD system for diagnosis of MCI due to AD using incomplete multi-modality imaging data

NASA Astrophysics Data System (ADS)

Liu, Xiaonan; Chen, Kewei; Wu, Teresa; Weidman, David; Lure, Fleming; Li, Jing

2018-02-01

Alzheimer's Disease (AD) is the most common cause of dementia and currently has no cure. Treatments targeting early stages of AD such as Mild Cognitive Impairment (MCI) may be most effective to deaccelerate AD, thus attracting increasing attention. However, MCI has substantial heterogeneity in that it can be caused by various underlying conditions, not only AD. To detect MCI due to AD, NIA-AA published updated consensus criteria in 2011, in which the use of multi-modality images was highlighted as one of the most promising methods. It is of great interest to develop a CAD system based on automatic, quantitative analysis of multi-modality images and machine learning algorithms to help physicians more adequately diagnose MCI due to AD. The challenge, however, is that multi-modality images are not universally available for many patients due to cost, access, safety, and lack of consent. We developed a novel Missing Modality Transfer Learning (MMTL) algorithm capable of utilizing whatever imaging modalities are available for an MCI patient to diagnose the patient's likelihood of MCI due to AD. Furthermore, we integrated MMTL with radiomics steps including image processing, feature extraction, and feature screening, and a post-processing for uncertainty quantification (UQ), and developed a CAD system called "ADMultiImg" to assist clinical diagnosis of MCI due to AD using multi-modality images together with patient demographic and genetic information. Tested on ADNI date, our system can generate a diagnosis with high accuracy even for patients with only partially available image modalities (AUC=0.94), and therefore may have broad clinical utility.

Novel multimodality segmentation using level sets and Jensen-Rényi divergence

DOE Office of Scientific and Technical Information (OSTI.GOV)

Markel, Daniel, E-mail: daniel.markel@mail.mcgill.ca; Zaidi, Habib; Geneva Neuroscience Center, Geneva University, CH-1205 Geneva

2013-12-15

Purpose: Positron emission tomography (PET) is playing an increasing role in radiotherapy treatment planning. However, despite progress, robust algorithms for PET and multimodal image segmentation are still lacking, especially if the algorithm were extended to image-guided and adaptive radiotherapy (IGART). This work presents a novel multimodality segmentation algorithm using the Jensen-Rényi divergence (JRD) to evolve the geometric level set contour. The algorithm offers improved noise tolerance which is particularly applicable to segmentation of regions found in PET and cone-beam computed tomography. Methods: A steepest gradient ascent optimization method is used in conjunction with the JRD and a level set activemore » contour to iteratively evolve a contour to partition an image based on statistical divergence of the intensity histograms. The algorithm is evaluated using PET scans of pharyngolaryngeal squamous cell carcinoma with the corresponding histological reference. The multimodality extension of the algorithm is evaluated using 22 PET/CT scans of patients with lung carcinoma and a physical phantom scanned under varying image quality conditions. Results: The average concordance index (CI) of the JRD segmentation of the PET images was 0.56 with an average classification error of 65%. The segmentation of the lung carcinoma images had a maximum diameter relative error of 63%, 19.5%, and 14.8% when using CT, PET, and combined PET/CT images, respectively. The estimated maximal diameters of the gross tumor volume (GTV) showed a high correlation with the macroscopically determined maximal diameters, with aR{sup 2} value of 0.85 and 0.88 using the PET and PET/CT images, respectively. Results from the physical phantom show that the JRD is more robust to image noise compared to mutual information and region growing. Conclusions: The JRD has shown improved noise tolerance compared to mutual information for the purpose of PET image segmentation. Presented is a flexible framework for multimodal image segmentation that can incorporate a large number of inputs efficiently for IGART.« less

Novel multimodality segmentation using level sets and Jensen-Rényi divergence.

PubMed

Markel, Daniel; Zaidi, Habib; El Naqa, Issam

2013-12-01

Positron emission tomography (PET) is playing an increasing role in radiotherapy treatment planning. However, despite progress, robust algorithms for PET and multimodal image segmentation are still lacking, especially if the algorithm were extended to image-guided and adaptive radiotherapy (IGART). This work presents a novel multimodality segmentation algorithm using the Jensen-Rényi divergence (JRD) to evolve the geometric level set contour. The algorithm offers improved noise tolerance which is particularly applicable to segmentation of regions found in PET and cone-beam computed tomography. A steepest gradient ascent optimization method is used in conjunction with the JRD and a level set active contour to iteratively evolve a contour to partition an image based on statistical divergence of the intensity histograms. The algorithm is evaluated using PET scans of pharyngolaryngeal squamous cell carcinoma with the corresponding histological reference. The multimodality extension of the algorithm is evaluated using 22 PET/CT scans of patients with lung carcinoma and a physical phantom scanned under varying image quality conditions. The average concordance index (CI) of the JRD segmentation of the PET images was 0.56 with an average classification error of 65%. The segmentation of the lung carcinoma images had a maximum diameter relative error of 63%, 19.5%, and 14.8% when using CT, PET, and combined PET/CT images, respectively. The estimated maximal diameters of the gross tumor volume (GTV) showed a high correlation with the macroscopically determined maximal diameters, with a R(2) value of 0.85 and 0.88 using the PET and PET/CT images, respectively. Results from the physical phantom show that the JRD is more robust to image noise compared to mutual information and region growing. The JRD has shown improved noise tolerance compared to mutual information for the purpose of PET image segmentation. Presented is a flexible framework for multimodal image segmentation that can incorporate a large number of inputs efficiently for IGART.

Beyond endoscopic assessment in inflammatory bowel disease: real-time histology of disease activity by non-linear multimodal imaging

NASA Astrophysics Data System (ADS)

Chernavskaia, Olga; Heuke, Sandro; Vieth, Michael; Friedrich, Oliver; Schürmann, Sebastian; Atreya, Raja; Stallmach, Andreas; Neurath, Markus F.; Waldner, Maximilian; Petersen, Iver; Schmitt, Michael; Bocklitz, Thomas; Popp, Jürgen

2016-07-01

Assessing disease activity is a prerequisite for an adequate treatment of inflammatory bowel diseases (IBD) such as Crohn’s disease and ulcerative colitis. In addition to endoscopic mucosal healing, histologic remission poses a promising end-point of IBD therapy. However, evaluating histological remission harbors the risk for complications due to the acquisition of biopsies and results in a delay of diagnosis because of tissue processing procedures. In this regard, non-linear multimodal imaging techniques might serve as an unparalleled technique that allows the real-time evaluation of microscopic IBD activity in the endoscopy unit. In this study, tissue sections were investigated using the non-linear multimodal microscopy combination of coherent anti-Stokes Raman scattering (CARS), two-photon excited auto fluorescence (TPEF) and second-harmonic generation (SHG). After the measurement a gold-standard assessment of histological indexes was carried out based on a conventional H&E stain. Subsequently, various geometry and intensity related features were extracted from the multimodal images. An optimized feature set was utilized to predict histological index levels based on a linear classifier. Based on the automated prediction, the diagnosis time interval is decreased. Therefore, non-linear multimodal imaging may provide a real-time diagnosis of IBD activity suited to assist clinical decision making within the endoscopy unit.

Multimodality imaging of adult gastric emergencies: A pictorial review

PubMed Central

Sunnapwar, Abhijit; Ojili, Vijayanadh; Katre, Rashmi; Shah, Hardik; Nagar, Arpit

2017-01-01

Acute gastric emergencies require urgent surgical or nonsurgical intervention because they are associated with high morbidity and mortality. Imaging plays an important role in diagnosis since the clinical symptoms are often nonspecific and radiologist may be the first one to suggest a diagnosis as the imaging findings are often characteristic. The purpose of this article is to provide a comprehensive review of multimodality imaging (plain radiograph, fluoroscopy, and computed tomography) of various life threatening gastric emergencies. PMID:28515579

Cross-Modality Image Synthesis via Weakly Coupled and Geometry Co-Regularized Joint Dictionary Learning.

PubMed

Huang, Yawen; Shao, Ling; Frangi, Alejandro F

2018-03-01

Multi-modality medical imaging is increasingly used for comprehensive assessment of complex diseases in either diagnostic examinations or as part of medical research trials. Different imaging modalities provide complementary information about living tissues. However, multi-modal examinations are not always possible due to adversary factors, such as patient discomfort, increased cost, prolonged scanning time, and scanner unavailability. In additionally, in large imaging studies, incomplete records are not uncommon owing to image artifacts, data corruption or data loss, which compromise the potential of multi-modal acquisitions. In this paper, we propose a weakly coupled and geometry co-regularized joint dictionary learning method to address the problem of cross-modality synthesis while considering the fact that collecting the large amounts of training data is often impractical. Our learning stage requires only a few registered multi-modality image pairs as training data. To employ both paired images and a large set of unpaired data, a cross-modality image matching criterion is proposed. Then, we propose a unified model by integrating such a criterion into the joint dictionary learning and the observed common feature space for associating cross-modality data for the purpose of synthesis. Furthermore, two regularization terms are added to construct robust sparse representations. Our experimental results demonstrate superior performance of the proposed model over state-of-the-art methods.

Application of Virtual Navigation with Multimodality Image Fusion in Foramen Ovale Cannulation.

PubMed

Qiu, Xixiong; Liu, Weizong; Zhang, Mingdong; Lin, Hengzhou; Zhou, Shoujun; Lei, Yi; Xia, Jun

2017-11-01

Idiopathic trigeminal neuralgia (ITN) can be effectively treated with radiofrequency thermocoagulation. However, this procedure requires cannulation of the foramen ovale, and conventional cannulation methods are associated with high failure rates. Multimodality imaging can improve the accuracy of cannulation because each imaging method can compensate for the drawbacks of the other. We aim to determine the feasibility and accuracy of percutaneous foramen ovale cannulation under the guidance of virtual navigation with multimodality image fusion in a self-designed anatomical model of human cadaveric heads. Five cadaveric head specimens were investigated in this study. Spiral computed tomography (CT) scanning clearly displayed the foramen ovale in all five specimens (10 foramina), which could not be visualized using two-dimensional ultrasound alone. The ultrasound and spiral CT images were fused, and percutaneous cannulation of the foramen ovale was performed under virtual navigation. After this, spiral CT scanning was immediately repeated to confirm the accuracy of the cannulation. Postprocedural spiral CT confirmed that the ultrasound and CT images had been successfully fused for all 10 foramina, which were accurately and successfully cannulated. The success rates of both image fusion and cannulation were 100%. Virtual navigation with multimodality image fusion can substantially facilitate foramen ovale cannulation and is worthy of clinical application. © 2017 American Academy of Pain Medicine. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com

XML-based scripting of multimodality image presentations in multidisciplinary clinical conferences

NASA Astrophysics Data System (ADS)

Ratib, Osman M.; Allada, Vivekanand; Dahlbom, Magdalena; Marcus, Phillip; Fine, Ian; Lapstra, Lorelle

2002-05-01

We developed a multi-modality image presentation software for display and analysis of images and related data from different imaging modalities. The software is part of a cardiac image review and presentation platform that supports integration of digital images and data from digital and analog media such as videotapes, analog x-ray films and 35 mm cine films. The software supports standard DICOM image files as well as AVI and PDF data formats. The system is integrated in a digital conferencing room that includes projections of digital and analog sources, remote videoconferencing capabilities, and an electronic whiteboard. The goal of this pilot project is to: 1) develop a new paradigm for image and data management for presentation in a clinically meaningful sequence adapted to case-specific scenarios, 2) design and implement a multi-modality review and conferencing workstation using component technology and customizable 'plug-in' architecture to support complex review and diagnostic tasks applicable to all cardiac imaging modalities and 3) develop an XML-based scripting model of image and data presentation for clinical review and decision making during routine clinical tasks and multidisciplinary clinical conferences.

Dual CARS and SHG image acquisition scheme that combines single central fiber and multimode fiber bundle to collect and differentiate backward and forward generated photons

PubMed Central

Weng, Sheng; Chen, Xu; Xu, Xiaoyun; Wong, Kelvin K.; Wong, Stephen T. C.

2016-01-01

In coherent anti-Stokes Raman scattering (CARS) and second harmonic generation (SHG) imaging, backward and forward generated photons exhibit different image patterns and thus capture salient intrinsic information of tissues from different perspectives. However, they are often mixed in collection using traditional image acquisition methods and thus are hard to interpret. We developed a multimodal scheme using a single central fiber and multimode fiber bundle to simultaneously collect and differentiate images formed by these two types of photons and evaluated the scheme in an endomicroscopy prototype. The ratio of these photons collected was calculated for the characterization of tissue regions with strong or weak epi-photon generation while different image patterns of these photons at different tissue depths were revealed. This scheme provides a new approach to extract and integrate information captured by backward and forward generated photons in dual CARS/SHG imaging synergistically for biomedical applications. PMID:27375938

Multimodal molecular 3D imaging for the tumoral volumetric distribution assessment of folate-based biosensors.

PubMed

Ramírez-Nava, Gerardo J; Santos-Cuevas, Clara L; Chairez, Isaac; Aranda-Lara, Liliana

2017-12-01

The aim of this study was to characterize the in vivo volumetric distribution of three folate-based biosensors by different imaging modalities (X-ray, fluorescence, Cerenkov luminescence, and radioisotopic imaging) through the development of a tridimensional image reconstruction algorithm. The preclinical and multimodal Xtreme imaging system, with a Multimodal Animal Rotation System (MARS), was used to acquire bidimensional images, which were processed to obtain the tridimensional reconstruction. Images of mice at different times (biosensor distribution) were simultaneously obtained from the four imaging modalities. The filtered back projection and inverse Radon transformation were used as main image-processing techniques. The algorithm developed in Matlab was able to calculate the volumetric profiles of 99m Tc-Folate-Bombesin (radioisotopic image), 177 Lu-Folate-Bombesin (Cerenkov image), and FolateRSense™ 680 (fluorescence image) in tumors and kidneys of mice, and no significant differences were detected in the volumetric quantifications among measurement techniques. The imaging tridimensional reconstruction algorithm can be easily extrapolated to different 2D acquisition-type images. This characteristic flexibility of the algorithm developed in this study is a remarkable advantage in comparison to similar reconstruction methods.

Tumor image signatures and habitats: a processing pipeline of multimodality metabolic and physiological images.

PubMed

You, Daekeun; Kim, Michelle M; Aryal, Madhava P; Parmar, Hemant; Piert, Morand; Lawrence, Theodore S; Cao, Yue

2018-01-01

To create tumor "habitats" from the "signatures" discovered from multimodality metabolic and physiological images, we developed a framework of a processing pipeline. The processing pipeline consists of six major steps: (1) creating superpixels as a spatial unit in a tumor volume; (2) forming a data matrix [Formula: see text] containing all multimodality image parameters at superpixels; (3) forming and clustering a covariance or correlation matrix [Formula: see text] of the image parameters to discover major image "signatures;" (4) clustering the superpixels and organizing the parameter order of the [Formula: see text] matrix according to the one found in step 3; (5) creating "habitats" in the image space from the superpixels associated with the "signatures;" and (6) pooling and clustering a matrix consisting of correlation coefficients of each pair of image parameters from all patients to discover subgroup patterns of the tumors. The pipeline was applied to a dataset of multimodality images in glioblastoma (GBM) first, which consisted of 10 image parameters. Three major image "signatures" were identified. The three major "habitats" plus their overlaps were created. To test generalizability of the processing pipeline, a second image dataset from GBM, acquired on the scanners different from the first one, was processed. Also, to demonstrate the clinical association of image-defined "signatures" and "habitats," the patterns of recurrence of the patients were analyzed together with image parameters acquired prechemoradiation therapy. An association of the recurrence patterns with image-defined "signatures" and "habitats" was revealed. These image-defined "signatures" and "habitats" can be used to guide stereotactic tissue biopsy for genetic and mutation status analysis and to analyze for prediction of treatment outcomes, e.g., patterns of failure.

Biological Parametric Mapping: A Statistical Toolbox for Multi-Modality Brain Image Analysis

PubMed Central

Casanova, Ramon; Ryali, Srikanth; Baer, Aaron; Laurienti, Paul J.; Burdette, Jonathan H.; Hayasaka, Satoru; Flowers, Lynn; Wood, Frank; Maldjian, Joseph A.

2006-01-01

In recent years multiple brain MR imaging modalities have emerged; however, analysis methodologies have mainly remained modality specific. In addition, when comparing across imaging modalities, most researchers have been forced to rely on simple region-of-interest type analyses, which do not allow the voxel-by-voxel comparisons necessary to answer more sophisticated neuroscience questions. To overcome these limitations, we developed a toolbox for multimodal image analysis called biological parametric mapping (BPM), based on a voxel-wise use of the general linear model. The BPM toolbox incorporates information obtained from other modalities as regressors in a voxel-wise analysis, thereby permitting investigation of more sophisticated hypotheses. The BPM toolbox has been developed in MATLAB with a user friendly interface for performing analyses, including voxel-wise multimodal correlation, ANCOVA, and multiple regression. It has a high degree of integration with the SPM (statistical parametric mapping) software relying on it for visualization and statistical inference. Furthermore, statistical inference for a correlation field, rather than a widely-used T-field, has been implemented in the correlation analysis for more accurate results. An example with in-vivo data is presented demonstrating the potential of the BPM methodology as a tool for multimodal image analysis. PMID:17070709

Recent Advances in Molecular, Multimodal and Theranostic Ultrasound Imaging

PubMed Central

Kiessling, Fabian; Fokong, Stanley; Bzyl, Jessica; Lederle, Wiltrud; Palmowski, Moritz; Lammers, Twan

2014-01-01

Ultrasound (US) imaging is an exquisite tool for the non-invasive and real-time diagnosis of many different diseases. In this context, US contrast agents can improve lesion delineation, characterization and therapy response evaluation. US contrast agents are usually micrometer-sized gas bubbles, stabilized with soft or hard shells. By conjugating antibodies to the microbubble (MB) surface, and by incorporating diagnostic agents, drugs or nucleic acids into or onto the MB shell, molecular, multimodal and theranostic MB can be generated. We here summarize recent advances in molecular, multimodal and theranostic US imaging, and introduce concepts how such advanced MB can be generated, applied and imaged. Examples are given for their use to image and treat oncological, cardiovascular and neurological diseases. Furthermore, we discuss for which therapeutic entities incorporation into (or conjugation to) MB is meaningful, and how US-mediated MB destruction can increase their extravasation, penetration, internalization and efficacy. PMID:24316070

The value of multimodality imaging in the investigation of a PSA recurrence after radical prostatectomy in the Irish hospital setting.

PubMed

McLoughlin, L C; Inder, S; Moran, D; O'Rourke, C; Manecksha, R P; Lynch, T H

2018-02-01

The diagnostic evaluation of a PSA recurrence after RP in the Irish hospital setting involves multimodality imaging with MRI, CT, and bone scanning, despite the low diagnostic yield from imaging at low PSA levels. We aim to investigate the value of multimodality imaging in PC patients after RP with a PSA recurrence. Forty-eight patients with a PSA recurrence after RP who underwent multimodality imaging were evaluated. Demographic data, postoperative PSA levels, and imaging studies performed at those levels were evaluated. Eight (21%) MRIs, 6 (33%) CTs, and 4 (9%) bone scans had PCa-specific findings. Three (12%) patients had a positive MRI with a PSA <1.0 ng/ml, while 5 (56%) were positive at PSA ≥1.1 ng/ml (p = 0.05). Zero patient had a positive CT TAP at a PSA level <1.0 ng/ml, while 5 (56%) were positive at levels ≥1.1 ng/ml (p = 0.03). Zero patient had a positive bone at PSA levels <1.0 ng/ml, while 4 (27%) were positive at levels ≥1.1 ng/ml (p = 0.01). The diagnostic yield from multimodality imaging, and isotope bone scanning in particular, in PSA levels <1.0 ng/ml, is low. There is a statistically significant increase in the frequency of positive findings on CT and bone scanning at PSA levels ≥1.1 ng/ml. MRI alone is of investigative value at PSA <1.0 ng/ml. The indication for CT, MRI, or isotope bone scanning should be carefully correlated with the clinical question and how it will affect further management.

Dense depth maps from correspondences derived from perceived motion

NASA Astrophysics Data System (ADS)

Kirby, Richard; Whitaker, Ross

2017-01-01

Many computer vision applications require finding corresponding points between images and using the corresponding points to estimate disparity. Today's correspondence finding algorithms primarily use image features or pixel intensities common between image pairs. Some 3-D computer vision applications, however, do not produce the desired results using correspondences derived from image features or pixel intensities. Two examples are the multimodal camera rig and the center region of a coaxial camera rig. We present an image correspondence finding technique that aligns pairs of image sequences using optical flow fields. The optical flow fields provide information about the structure and motion of the scene, which are not available in still images but can be used in image alignment. We apply the technique to a dual focal length stereo camera rig consisting of a visible light-infrared camera pair and to a coaxial camera rig. We test our method on real image sequences and compare our results with the state-of-the-art multimodal and structure from motion (SfM) algorithms. Our method produces more accurate depth and scene velocity reconstruction estimates than the state-of-the-art multimodal and SfM algorithms.

Fast and robust multimodal image registration using a local derivative pattern.

PubMed

Jiang, Dongsheng; Shi, Yonghong; Chen, Xinrong; Wang, Manning; Song, Zhijian

2017-02-01

Deformable multimodal image registration, which can benefit radiotherapy and image guided surgery by providing complementary information, remains a challenging task in the medical image analysis field due to the difficulty of defining a proper similarity measure. This article presents a novel, robust and fast binary descriptor, the discriminative local derivative pattern (dLDP), which is able to encode images of different modalities into similar image representations. dLDP calculates a binary string for each voxel according to the pattern of intensity derivatives in its neighborhood. The descriptor similarity is evaluated using the Hamming distance, which can be efficiently computed, instead of conventional L1 or L2 norms. For the first time, we validated the effectiveness and feasibility of the local derivative pattern for multimodal deformable image registration with several multi-modal registration applications. dLDP was compared with three state-of-the-art methods in artificial image and clinical settings. In the experiments of deformable registration between different magnetic resonance imaging (MRI) modalities from BrainWeb, between computed tomography and MRI images from patient data, and between MRI and ultrasound images from BITE database, we show our method outperforms localized mutual information and entropy images in terms of both accuracy and time efficiency. We have further validated dLDP for the deformable registration of preoperative MRI and three-dimensional intraoperative ultrasound images. Our results indicate that dLDP reduces the average mean target registration error from 4.12 mm to 2.30 mm. This accuracy is statistically equivalent to the accuracy of the state-of-the-art methods in the study; however, in terms of computational complexity, our method significantly outperforms other methods and is even comparable to the sum of the absolute difference. The results reveal that dLDP can achieve superior performance regarding both accuracy and time efficiency in general multimodal image registration. In addition, dLDP also indicates the potential for clinical ultrasound guided intervention. © 2016 The Authors. Medical Physics published by Wiley Periodicals, Inc. on behalf of American Association of Physicists in Medicine.

Gadolinium-Conjugated Gold Nanoshells for Multimodal Diagnostic Imaging and Photothermal Cancer Therapy

PubMed Central

Coughlin, Andrew J.; Ananta, Jeyarama S.; Deng, Nanfu; Larina, Irina V.; Decuzzi, Paolo

2014-01-01

Multimodal imaging offers the potential to improve diagnosis and enhance the specificity of photothermal cancer therapy. Toward this goal, we have engineered gadolinium-conjugated gold nanoshells and demonstrated that they enhance contrast for magnetic resonance imaging, X-Ray, optical coherence tomography, reflectance confocal microscopy, and two-photon luminescence. Additionally, these particles effectively convert near-infrared light to heat, which can be used to ablate cancer cells. Ultimately, these studies demonstrate the potential of gadolinium-nanoshells for image-guided photothermal ablation. PMID:24115690

Intraoperative imaging-guided cancer surgery: from current fluorescence molecular imaging methods to future multi-modality imaging technology.

PubMed

Chi, Chongwei; Du, Yang; Ye, Jinzuo; Kou, Deqiang; Qiu, Jingdan; Wang, Jiandong; Tian, Jie; Chen, Xiaoyuan

2014-01-01

Cancer is a major threat to human health. Diagnosis and treatment using precision medicine is expected to be an effective method for preventing the initiation and progression of cancer. Although anatomical and functional imaging techniques such as radiography, computed tomography (CT), magnetic resonance imaging (MRI) and positron emission tomography (PET) have played an important role for accurate preoperative diagnostics, for the most part these techniques cannot be applied intraoperatively. Optical molecular imaging is a promising technique that provides a high degree of sensitivity and specificity in tumor margin detection. Furthermore, existing clinical applications have proven that optical molecular imaging is a powerful intraoperative tool for guiding surgeons performing precision procedures, thus enabling radical resection and improved survival rates. However, detection depth limitation exists in optical molecular imaging methods and further breakthroughs from optical to multi-modality intraoperative imaging methods are needed to develop more extensive and comprehensive intraoperative applications. Here, we review the current intraoperative optical molecular imaging technologies, focusing on contrast agents and surgical navigation systems, and then discuss the future prospects of multi-modality imaging technology for intraoperative imaging-guided cancer surgery.

Intraoperative Imaging-Guided Cancer Surgery: From Current Fluorescence Molecular Imaging Methods to Future Multi-Modality Imaging Technology

PubMed Central

Chi, Chongwei; Du, Yang; Ye, Jinzuo; Kou, Deqiang; Qiu, Jingdan; Wang, Jiandong; Tian, Jie; Chen, Xiaoyuan

2014-01-01

Cancer is a major threat to human health. Diagnosis and treatment using precision medicine is expected to be an effective method for preventing the initiation and progression of cancer. Although anatomical and functional imaging techniques such as radiography, computed tomography (CT), magnetic resonance imaging (MRI) and positron emission tomography (PET) have played an important role for accurate preoperative diagnostics, for the most part these techniques cannot be applied intraoperatively. Optical molecular imaging is a promising technique that provides a high degree of sensitivity and specificity in tumor margin detection. Furthermore, existing clinical applications have proven that optical molecular imaging is a powerful intraoperative tool for guiding surgeons performing precision procedures, thus enabling radical resection and improved survival rates. However, detection depth limitation exists in optical molecular imaging methods and further breakthroughs from optical to multi-modality intraoperative imaging methods are needed to develop more extensive and comprehensive intraoperative applications. Here, we review the current intraoperative optical molecular imaging technologies, focusing on contrast agents and surgical navigation systems, and then discuss the future prospects of multi-modality imaging technology for intraoperative imaging-guided cancer surgery. PMID:25250092

Introduction of a standardized multimodality image protocol for navigation-guided surgery of suspected low-grade gliomas.

PubMed

Mert, Aygül; Kiesel, Barbara; Wöhrer, Adelheid; Martínez-Moreno, Mauricio; Minchev, Georgi; Furtner, Julia; Knosp, Engelbert; Wolfsberger, Stefan; Widhalm, Georg

2015-01-01

OBJECT Surgery of suspected low-grade gliomas (LGGs) poses a special challenge for neurosurgeons due to their diffusely infiltrative growth and histopathological heterogeneity. Consequently, neuronavigation with multimodality imaging data, such as structural and metabolic data, fiber tracking, and 3D brain visualization, has been proposed to optimize surgery. However, currently no standardized protocol has been established for multimodality imaging data in modern glioma surgery. The aim of this study was therefore to define a specific protocol for multimodality imaging and navigation for suspected LGG. METHODS Fifty-one patients who underwent surgery for a diffusely infiltrating glioma with nonsignificant contrast enhancement on MRI and available multimodality imaging data were included. In the first 40 patients with glioma, the authors retrospectively reviewed the imaging data, including structural MRI (contrast-enhanced T1-weighted, T2-weighted, and FLAIR sequences), metabolic images derived from PET, or MR spectroscopy chemical shift imaging, fiber tracking, and 3D brain surface/vessel visualization, to define standardized image settings and specific indications for each imaging modality. The feasibility and surgical relevance of this new protocol was subsequently prospectively investigated during surgery with the assistance of an advanced electromagnetic navigation system in the remaining 11 patients. Furthermore, specific surgical outcome parameters, including the extent of resection, histological analysis of the metabolic hotspot, presence of a new postoperative neurological deficit, and intraoperative accuracy of 3D brain visualization models, were assessed in each of these patients. RESULTS After reviewing these first 40 cases of glioma, the authors defined a specific protocol with standardized image settings and specific indications that allows for optimal and simultaneous visualization of structural and metabolic data, fiber tracking, and 3D brain visualization. This new protocol was feasible and was estimated to be surgically relevant during navigation-guided surgery in all 11 patients. According to the authors' predefined surgical outcome parameters, they observed a complete resection in all resectable gliomas (n = 5) by using contour visualization with T2-weighted or FLAIR images. Additionally, tumor tissue derived from the metabolic hotspot showed the presence of malignant tissue in all WHO Grade III or IV gliomas (n = 5). Moreover, no permanent postoperative neurological deficits occurred in any of these patients, and fiber tracking and/or intraoperative monitoring were applied during surgery in the vast majority of cases (n = 10). Furthermore, the authors found a significant intraoperative topographical correlation of 3D brain surface and vessel models with gyral anatomy and superficial vessels. Finally, real-time navigation with multimodality imaging data using the advanced electromagnetic navigation system was found to be useful for precise guidance to surgical targets, such as the tumor margin or the metabolic hotspot. CONCLUSIONS In this study, the authors defined a specific protocol for multimodality imaging data in suspected LGGs, and they propose the application of this new protocol for advanced navigation-guided procedures optimally in conjunction with continuous electromagnetic instrument tracking to optimize glioma surgery.

Nanoengineered multimodal contrast agent for medical image guidance

NASA Astrophysics Data System (ADS)

Perkins, Gregory J.; Zheng, Jinzi; Brock, Kristy; Allen, Christine; Jaffray, David A.

2005-04-01

Multimodality imaging has gained momentum in radiation therapy planning and image-guided treatment delivery. Specifically, computed tomography (CT) and magnetic resonance (MR) imaging are two complementary imaging modalities often utilized in radiation therapy for visualization of anatomical structures for tumour delineation and accurate registration of image data sets for volumetric dose calculation. The development of a multimodal contrast agent for CT and MR with prolonged in vivo residence time would provide long-lasting spatial and temporal correspondence of the anatomical features of interest, and therefore facilitate multimodal image registration, treatment planning and delivery. The multimodal contrast agent investigated consists of nano-sized stealth liposomes encapsulating conventional iodine and gadolinium-based contrast agents. The average loading achieved was 33.5 +/- 7.1 mg/mL of iodine for iohexol and 9.8 +/- 2.0 mg/mL of gadolinium for gadoteridol. The average liposome diameter was 46.2 +/- 13.5 nm. The system was found to be stable in physiological buffer over a 15-day period, releasing 11.9 +/- 1.1% and 11.2 +/- 0.9% of the total amounts of iohexol and gadoteridol loaded, respectively. 200 minutes following in vivo administration, the contrast agent maintained a relative contrast enhancement of 81.4 +/- 13.05 differential Hounsfield units (ΔHU) in CT (40% decrease from the peak signal value achieved 3 minutes post-injection) and 731.9 +/- 144.2 differential signal intensity (ΔSI) in MR (46% decrease from the peak signal value achieved 3 minutes post-injection) in the blood (aorta), a relative contrast enhancement of 38.0 +/- 5.1 ΔHU (42% decrease from the peak signal value achieved 3 minutes post-injection) and 178.6 +/- 41.4 ΔSI (62% decrease from the peak signal value achieved 3 minutes post-injection) in the liver (parenchyma), a relative contrast enhancement of 9.1 +/- 1.7 ΔHU (94% decrease from the peak signal value achieved 3 minutes post-injection) and 461.7 +/- 78.1 ΔSI (60% decrease from the peak signal value achieved 5 minutes post-injection) in the kidney (cortex) of a New Zealand white rabbit. This multimodal contrast agent, with prolonged in vivo residence time and imaging efficacy, has the potential to bring about improvements in the fields of medical imaging and radiation therapy, particularly for image registration and guidance.

Multimodal Image-Based Virtual Reality Presurgical Simulation and Evaluation for Trigeminal Neuralgia and Hemifacial Spasm.

PubMed

Yao, Shujing; Zhang, Jiashu; Zhao, Yining; Hou, Yuanzheng; Xu, Xinghua; Zhang, Zhizhong; Kikinis, Ron; Chen, Xiaolei

2018-05-01

To address the feasibility and predictive value of multimodal image-based virtual reality in detecting and assessing features of neurovascular confliction (NVC), particularly regarding the detection of offending vessels, degree of compression exerted on the nerve root, in patients who underwent microvascular decompression for nonlesional trigeminal neuralgia and hemifacial spasm (HFS). This prospective study includes 42 consecutive patients who underwent microvascular decompression for classic primary trigeminal neuralgia or HFS. All patients underwent preoperative 1.5-T magnetic resonance imaging (MRI) with T2-weighted three-dimensional (3D) sampling perfection with application-optimized contrasts by using different flip angle evolutions, 3D time-of-flight magnetic resonance angiography, and 3D T1-weighted gadolinium-enhanced sequences in combination, whereas 2 patients underwent extra experimental preoperative 7.0-T MRI scans with the same imaging protocol. Multimodal MRIs were then coregistered with open-source software 3D Slicer, followed by 3D image reconstruction to generate virtual reality (VR) images for detection of possible NVC in the cerebellopontine angle. Evaluations were performed by 2 reviewers and compared with the intraoperative findings. For detection of NVC, multimodal image-based VR sensitivity was 97.6% (40/41) and specificity was 100% (1/1). Compared with the intraoperative findings, the κ coefficients for predicting the offending vessel and the degree of compression were >0.75 (P < 0.001). The 7.0-T scans have a clearer view of vessels in the cerebellopontine angle, which may have significant impact on detection of small-caliber offending vessels with relatively slow flow speed in cases of HFS. Multimodal image-based VR using 3D sampling perfection with application-optimized contrasts by using different flip angle evolutions in combination with 3D time-of-flight magnetic resonance angiography sequences proved to be reliable in detecting NVC and in predicting the degree of root compression. The VR image-based simulation correlated well with the real surgical view. Copyright © 2018 Elsevier Inc. All rights reserved.

Multimodal Research: Addressing the Complexity of Multimodal Environments and the Challenges for CALL

ERIC Educational Resources Information Center

Tan, Sabine; O'Halloran, Kay L.; Wignell, Peter

2016-01-01

Multimodality, the study of the interaction of language with other semiotic resources such as images and sound resources, has significant implications for computer assisted language learning (CALL) with regards to understanding the impact of digital environments on language teaching and learning. In this paper, we explore recent manifestations of…

Feature-based Alignment of Volumetric Multi-modal Images

PubMed Central

Toews, Matthew; Zöllei, Lilla; Wells, William M.

2014-01-01

This paper proposes a method for aligning image volumes acquired from different imaging modalities (e.g. MR, CT) based on 3D scale-invariant image features. A novel method for encoding invariant feature geometry and appearance is developed, based on the assumption of locally linear intensity relationships, providing a solution to poor repeatability of feature detection in different image modalities. The encoding method is incorporated into a probabilistic feature-based model for multi-modal image alignment. The model parameters are estimated via a group-wise alignment algorithm, that iteratively alternates between estimating a feature-based model from feature data, then realigning feature data to the model, converging to a stable alignment solution with few pre-processing or pre-alignment requirements. The resulting model can be used to align multi-modal image data with the benefits of invariant feature correspondence: globally optimal solutions, high efficiency and low memory usage. The method is tested on the difficult RIRE data set of CT, T1, T2, PD and MP-RAGE brain images of subjects exhibiting significant inter-subject variability due to pathology. PMID:24683955

Rapid Screening of Cancer Margins in Tissue with Multimodal Confocal Microscopy

PubMed Central

Gareau, Daniel S.; Jeon, Hana; Nehal, Kishwer S.; Rajadhyaksha, Milind

2012-01-01

Background Complete and accurate excision of cancer is guided by the examination of histopathology. However, preparation of histopathology is labor intensive and slow, leading to insufficient sampling of tissue and incomplete and/or inaccurate excision of margins. We demonstrate the potential utility of multimodal confocal mosaicing microscopy for rapid screening of cancer margins, directly in fresh surgical excisions, without the need for conventional embedding, sectioning or processing. Materials/Methods A multimodal confocal mosaicing microscope was developed to image basal cell carcinoma margins in surgical skin excisions, with resolution that shows nuclear detail. Multimodal contrast is with fluorescence for imaging nuclei and reflectance for cellular cytoplasm and dermal collagen. Thirtyfive excisions of basal cell carcinomas from Mohs surgery were imaged, and the mosaics analyzed by comparison to the corresponding frozen pathology. Results Confocal mosaics are produced in about 9 minutes, displaying tissue in fields-of-view of 12 mm with 2X magnification. A digital staining algorithm transforms black and white contrast to purple and pink, which simulates the appearance of standard histopathology. Mosaicing enables rapid digital screening, which mimics the examination of histopathology. Conclusions Multimodal confocal mosaicing microscopy offers a technology platform to potentially enable real-time pathology at the bedside. The imaging may serve as an adjunct to conventional histopathology, to expedite screening of margins and guide surgery toward more complete and accurate excision of cancer. PMID:22721570

Development of a multi-scale and multi-modality imaging system to characterize tumours and their microenvironment in vivo

NASA Astrophysics Data System (ADS)

Rouffiac, Valérie; Ser-Leroux, Karine; Dugon, Emilie; Leguerney, Ingrid; Polrot, Mélanie; Robin, Sandra; Salomé-Desnoulez, Sophie; Ginefri, Jean-Christophe; Sebrié, Catherine; Laplace-Builhé, Corinne

2015-03-01

In vivo high-resolution imaging of tumor development is possible through dorsal skinfold chamber implantable on mice model. However, current intravital imaging systems are weakly tolerated along time by mice and do not allow multimodality imaging. Our project aims to develop a new chamber for: 1- long-term micro/macroscopic visualization of tumor (vascular and cellular compartments) and tissue microenvironment; and 2- multimodality imaging (photonic, MRI and sonography). Our new experimental device was patented in March 2014 and was primarily assessed on 75 mouse engrafted with 4T1-Luc tumor cell line, and validated in confocal and multiphoton imaging after staining the mice vasculature using Dextran 155KDa-TRITC or Dextran 2000kDa-FITC. Simultaneously, a universal stage was designed for optimal removal of respiratory and cardiac artifacts during microscopy assays. Experimental results from optical, ultrasound (Bmode and pulse subtraction mode) and MRI imaging (anatomic sequences) showed that our patented design, unlike commercial devices, improves longitudinal monitoring over several weeks (35 days on average against 12 for the commercial chamber) and allows for a better characterization of the early and late tissue alterations due to tumour development. We also demonstrated the compatibility for multimodality imaging and the increase of mice survival was by a factor of 2.9, with our new skinfold chamber. Current developments include: 1- defining new procedures for multi-labelling of cells and tissue (screening of fluorescent molecules and imaging protocols); 2- developing ultrasound and MRI imaging procedures with specific probes; 3- correlating optical/ultrasound/MRI data for a complete mapping of tumour development and microenvironment.

Deep Convolutional Neural Networks for Multi-Modality Isointense Infant Brain Image Segmentation

PubMed Central

Zhang, Wenlu; Li, Rongjian; Deng, Houtao; Wang, Li; Lin, Weili; Ji, Shuiwang; Shen, Dinggang

2015-01-01

The segmentation of infant brain tissue images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) plays an important role in studying early brain development in health and disease. In the isointense stage (approximately 6–8 months of age), WM and GM exhibit similar levels of intensity in both T1 and T2 MR images, making the tissue segmentation very challenging. Only a small number of existing methods have been designed for tissue segmentation in this isointense stage; however, they only used a single T1 or T2 images, or the combination of T1 and T2 images. In this paper, we propose to use deep convolutional neural networks (CNNs) for segmenting isointense stage brain tissues using multi-modality MR images. CNNs are a type of deep models in which trainable filters and local neighborhood pooling operations are applied alternatingly on the raw input images, resulting in a hierarchy of increasingly complex features. Specifically, we used multimodality information from T1, T2, and fractional anisotropy (FA) images as inputs and then generated the segmentation maps as outputs. The multiple intermediate layers applied convolution, pooling, normalization, and other operations to capture the highly nonlinear mappings between inputs and outputs. We compared the performance of our approach with that of the commonly used segmentation methods on a set of manually segmented isointense stage brain images. Results showed that our proposed model significantly outperformed prior methods on infant brain tissue segmentation. In addition, our results indicated that integration of multi-modality images led to significant performance improvement. PMID:25562829

Multimodal Imaging in Klippel-Trénaunay-Weber Syndrome: Clinical Photography, Computed Tomoangiography, Infrared Thermography, and 99mTc-Phytate Lymphoscintigraphy.

PubMed

Kim, Su Wan; Song, Heesung

2017-12-01

We report the case of a 19-year-old man who presented with a 12-year history of progressive fatigue, feeling hot, excessive sweating, and numbness in the left arm. He had undergone multimodal imaging and was diagnosed as having Klippel-Trénaunay-Weber syndrome (KTWS). This is a rare congenital disease, defined by combinations of nevus flammeus, venous and lymphatic malformation, and hypertrophy of the affected limbs. Lower extremities are affected mostly. Conventional modalities for evaluating KTWS are ultrasonography, CT, MRI, lymphoscintigraphy, and angiography. There are few reports on multimodal imaging of upper extremities of KTWS patients, and this is the first report of an infrared thermography in KTWS.

High-resolution multimodal clinical multiphoton tomography of skin

NASA Astrophysics Data System (ADS)

König, Karsten

2011-03-01

This review focuses on multimodal multiphoton tomography based on near infrared femtosecond lasers. Clinical multiphoton tomographs for 3D high-resolution in vivo imaging have been placed into the market several years ago. The second generation of this Prism-Award winning High-Tech skin imaging tool (MPTflex) was introduced in 2010. The same year, the world's first clinical CARS studies have been performed with a hybrid multimodal multiphoton tomograph. In particular, non-fluorescent lipids and water as well as mitochondrial fluorescent NAD(P)H, fluorescent elastin, keratin, and melanin as well as SHG-active collagen has been imaged with submicron resolution in patients suffering from psoriasis. Further multimodal approaches include the combination of multiphoton tomographs with low-resolution wide-field systems such as ultrasound, optoacoustical, OCT, and dermoscopy systems. Multiphoton tomographs are currently employed in Australia, Japan, the US, and in several European countries for early diagnosis of skin cancer, optimization of treatment strategies, and cosmetic research including long-term testing of sunscreen nanoparticles as well as anti-aging products.

Ultrasmall biomolecule-anchored hybrid GdVO4 nanophosphors as a metabolizable multimodal bioimaging contrast agent.

PubMed

Dong, Kai; Ju, Enguo; Liu, Jianhua; Han, Xueli; Ren, Jinsong; Qu, Xiaogang

2014-10-21

Multimodal molecular imaging has recently attracted much attention on disease diagnostics by taking advantage of individual imaging modalities. Herein, we have demonstrated a new paradigm for multimodal bioimaging based on amino acids-anchored ultrasmall lanthanide-doped GdVO4 nanoprobes. On the merit of special metal-cation complexation and abundant functional groups, these amino acids-anchored nanoprobes showed high colloidal stability and excellent dispersibility. Additionally, due to typical paramagnetic behaviour, high X-ray mass absorption coefficient and strong fluorescence, these nanoprobes would provide a unique opportunity to develop multifunctional probes for MRI, CT and luminescence imaging. More importantly, the small size and biomolecular coatings endow the nanoprobes with effective metabolisability and high biocompatibility. With the superior stability, high biocompatibility, effective metabolisability and excellent contrast performance, amino acids-capped GdVO4:Eu(3+) nanocastings are a promising candidate as multimodal contrast agents and would bring more opportunities for biological and medical applications with further modifications.

Radionuclide Myocardial Perfusion Imaging for the Evaluation of Patients With Known or Suspected Coronary Artery Disease in the Era of Multimodality Cardiovascular Imaging

PubMed Central

Taqueti, Viviany R.; Di Carli, Marcelo F.

2018-01-01

Over the last several decades, radionuclide myocardial perfusion imaging (MPI) with single photon emission tomography and positron emission tomography has been a mainstay for the evaluation of patients with known or suspected coronary artery disease (CAD). More recently, technical advances in separate and complementary imaging modalities including coronary computed tomography angiography, computed tomography perfusion, cardiac magnetic resonance imaging, and contrast stress echocardiography have expanded the toolbox of diagnostic testing for cardiac patients. While the growth of available technologies has heralded an exciting era of multimodality cardiovascular imaging, coordinated and dispassionate utilization of these techniques is needed to implement the right test for the right patient at the right time, a promise of “precision medicine.” In this article, we review the maturing role of MPI in the current era of multimodality cardiovascular imaging, particularly in the context of recent advances in myocardial blood flow quantitation, and as applied to the evaluation of patients with known or suspected CAD. PMID:25770849

Integrated scanning laser ophthalmoscopy and optical coherence tomography for quantitative multimodal imaging of retinal degeneration and autofluorescence

NASA Astrophysics Data System (ADS)

Issaei, Ali; Szczygiel, Lukasz; Hossein-Javaheri, Nima; Young, Mei; Molday, L. L.; Molday, R. S.; Sarunic, M. V.

2011-03-01

Scanning Laser Ophthalmoscopy (SLO) and Coherence Tomography (OCT) are complimentary retinal imaging modalities. Integration of SLO and OCT allows for both fluorescent detection and depth- resolved structural imaging of the retinal cell layers to be performed in-vivo. System customization is required to image rodents used in medical research by vision scientists. We are investigating multimodal SLO/OCT imaging of a rodent model of Stargardt's Macular Dystrophy which is characterized by retinal degeneration and accumulation of toxic autofluorescent lipofuscin deposits. Our new findings demonstrate the ability to track fundus autofluorescence and retinal degeneration concurrently.

Multimodal Image Alignment via Linear Mapping between Feature Modalities.

PubMed

Jiang, Yanyun; Zheng, Yuanjie; Hou, Sujuan; Chang, Yuchou; Gee, James

2017-01-01

We propose a novel landmark matching based method for aligning multimodal images, which is accomplished uniquely by resolving a linear mapping between different feature modalities. This linear mapping results in a new measurement on similarity of images captured from different modalities. In addition, our method simultaneously solves this linear mapping and the landmark correspondences by minimizing a convex quadratic function. Our method can estimate complex image relationship between different modalities and nonlinear nonrigid spatial transformations even in the presence of heavy noise, as shown in our experiments carried out by using a variety of image modalities.

Medical image registration based on normalized multidimensional mutual information

NASA Astrophysics Data System (ADS)

Li, Qi; Ji, Hongbing; Tong, Ming

2009-10-01

Registration of medical images is an essential research topic in medical image processing and applications, and especially a preliminary and key step for multimodality image fusion. This paper offers a solution to medical image registration based on normalized multi-dimensional mutual information. Firstly, affine transformation with translational and rotational parameters is applied to the floating image. Then ordinal features are extracted by ordinal filters with different orientations to represent spatial information in medical images. Integrating ordinal features with pixel intensities, the normalized multi-dimensional mutual information is defined as similarity criterion to register multimodality images. Finally the immune algorithm is used to search registration parameters. The experimental results demonstrate the effectiveness of the proposed registration scheme.

Use of multimodality imaging and artificial intelligence for diagnosis and prognosis of early stages of Alzheimer's disease.

PubMed

Liu, Xiaonan; Chen, Kewei; Wu, Teresa; Weidman, David; Lure, Fleming; Li, Jing

2018-04-01

Alzheimer's disease (AD) is a major neurodegenerative disease and the most common cause of dementia. Currently, no treatment exists to slow down or stop the progression of AD. There is converging belief that disease-modifying treatments should focus on early stages of the disease, that is, the mild cognitive impairment (MCI) and preclinical stages. Making a diagnosis of AD and offering a prognosis (likelihood of converting to AD) at these early stages are challenging tasks but possible with the help of multimodality imaging, such as magnetic resonance imaging (MRI), fluorodeoxyglucose (FDG)-positron emission topography (PET), amyloid-PET, and recently introduced tau-PET, which provides different but complementary information. This article is a focused review of existing research in the recent decade that used statistical machine learning and artificial intelligence methods to perform quantitative analysis of multimodality image data for diagnosis and prognosis of AD at the MCI or preclinical stages. We review the existing work in 3 subareas: diagnosis, prognosis, and methods for handling modality-wise missing data-a commonly encountered problem when using multimodality imaging for prediction or classification. Factors contributing to missing data include lack of imaging equipment, cost, difficulty of obtaining patient consent, and patient drop-off (in longitudinal studies). Finally, we summarize our major findings and provide some recommendations for potential future research directions. Copyright © 2018 Elsevier Inc. All rights reserved.

Overlapping-image multimode interference couplers with a reduced number of self-images for uniform and nonuniform power splitting

NASA Astrophysics Data System (ADS)

Bachmann, M.; Besse, P. A.; Melchior, H.

1995-10-01

Overlapping-image multimode interference (MMI) couplers, a new class of devices, permit uniform and nonuniform power splitting. A theoretical description directly relates coupler geometry to image intensities, positions, and phases. Among many possibilities of nonuniform power splitting, examples of 1 \\times 2 couplers with ratios of 15:85 and 28:72 are given. An analysis of uniform power splitters includes the well-known 2 \\times N and 1 \\times N MMI couplers. Applications of MMI couplers include mode filters, mode splitters-combiners, and mode converters.

Magnetic nanobubbles with potential for targeted drug delivery and trimodal imaging in breast cancer: an in vitro study.

PubMed

Song, Weixiang; Luo, Yindeng; Zhao, Yajing; Liu, Xinjie; Zhao, Jiannong; Luo, Jie; Zhang, Qunxia; Ran, Haitao; Wang, Zhigang; Guo, Dajing

2017-05-01

The aim of this study was to improve tumor-targeted therapy for breast cancer by designing magnetic nanobubbles with the potential for targeted drug delivery and multimodal imaging. Herceptin-decorated and ultrasmall superparamagnetic iron oxide (USPIO)/paclitaxel (PTX)-embedded nanobubbles (PTX-USPIO-HER-NBs) were manufactured by combining a modified double-emulsion evaporation process with carbodiimide technique. PTX-USPIO-HER-NBs were examined for characterization, specific cell-targeting ability and multimodal imaging. PTX-USPIO-HER-NBs exhibited excellent entrapment efficiency of Herceptin/PTX/USPIO and showed greater cytotoxic effects than other delivery platforms. Low-frequency ultrasound triggered accelerated PTX release. Moreover, the magnetic nanobubbles were able to enhance ultrasound, magnetic resonance and photoacoustics trimodal imaging. These results suggest that PTX-USPIO-HER-NBs have potential as a multimodal contrast agent and as a system for ultrasound-triggered drug release in breast cancer.

Spiking Cortical Model Based Multimodal Medical Image Fusion by Combining Entropy Information with Weber Local Descriptor

PubMed Central

Zhang, Xuming; Ren, Jinxia; Huang, Zhiwen; Zhu, Fei

2016-01-01

Multimodal medical image fusion (MIF) plays an important role in clinical diagnosis and therapy. Existing MIF methods tend to introduce artifacts, lead to loss of image details or produce low-contrast fused images. To address these problems, a novel spiking cortical model (SCM) based MIF method has been proposed in this paper. The proposed method can generate high-quality fused images using the weighting fusion strategy based on the firing times of the SCM. In the weighting fusion scheme, the weight is determined by combining the entropy information of pulse outputs of the SCM with the Weber local descriptor operating on the firing mapping images produced from the pulse outputs. The extensive experiments on multimodal medical images show that compared with the numerous state-of-the-art MIF methods, the proposed method can preserve image details very well and avoid the introduction of artifacts effectively, and thus it significantly improves the quality of fused images in terms of human vision and objective evaluation criteria such as mutual information, edge preservation index, structural similarity based metric, fusion quality index, fusion similarity metric and standard deviation. PMID:27649190

Spiking Cortical Model Based Multimodal Medical Image Fusion by Combining Entropy Information with Weber Local Descriptor.

PubMed

Zhang, Xuming; Ren, Jinxia; Huang, Zhiwen; Zhu, Fei

2016-09-15

Multimodal medical image fusion (MIF) plays an important role in clinical diagnosis and therapy. Existing MIF methods tend to introduce artifacts, lead to loss of image details or produce low-contrast fused images. To address these problems, a novel spiking cortical model (SCM) based MIF method has been proposed in this paper. The proposed method can generate high-quality fused images using the weighting fusion strategy based on the firing times of the SCM. In the weighting fusion scheme, the weight is determined by combining the entropy information of pulse outputs of the SCM with the Weber local descriptor operating on the firing mapping images produced from the pulse outputs. The extensive experiments on multimodal medical images show that compared with the numerous state-of-the-art MIF methods, the proposed method can preserve image details very well and avoid the introduction of artifacts effectively, and thus it significantly improves the quality of fused images in terms of human vision and objective evaluation criteria such as mutual information, edge preservation index, structural similarity based metric, fusion quality index, fusion similarity metric and standard deviation.

Exogenous Molecular Probes for Targeted Imaging in Cancer: Focus on Multi-modal Imaging

PubMed Central

Joshi, Bishnu P.; Wang, Thomas D.

2010-01-01

Cancer is one of the major causes of mortality and morbidity in our healthcare system. Molecular imaging is an emerging methodology for the early detection of cancer, guidance of therapy, and monitoring of response. The development of new instruments and exogenous molecular probes that can be labeled for multi-modality imaging is critical to this process. Today, molecular imaging is at a crossroad, and new targeted imaging agents are expected to broadly expand our ability to detect and manage cancer. This integrated imaging strategy will permit clinicians to not only localize lesions within the body but also to manage their therapy by visualizing the expression and activity of specific molecules. This information is expected to have a major impact on drug development and understanding of basic cancer biology. At this time, a number of molecular probes have been developed by conjugating various labels to affinity ligands for targeting in different imaging modalities. This review will describe the current status of exogenous molecular probes for optical, scintigraphic, MRI and ultrasound imaging platforms. Furthermore, we will also shed light on how these techniques can be used synergistically in multi-modal platforms and how these techniques are being employed in current research. PMID:22180839

Multimodal Task-Driven Dictionary Learning for Image Classification

DTIC Science & Technology

2015-12-18

1 Multimodal Task-Driven Dictionary Learning for Image Classification Soheil Bahrampour, Student Member, IEEE, Nasser M. Nasrabadi, Fellow, IEEE...Asok Ray, Fellow, IEEE, and W. Kenneth Jenkins, Life Fellow, IEEE Abstract— Dictionary learning algorithms have been suc- cessfully used for both...reconstructive and discriminative tasks, where an input signal is represented with a sparse linear combination of dictionary atoms. While these methods are

Learning of Multimodal Representations With Random Walks on the Click Graph.

PubMed

Wu, Fei; Lu, Xinyan; Song, Jun; Yan, Shuicheng; Zhang, Zhongfei Mark; Rui, Yong; Zhuang, Yueting

2016-02-01

In multimedia information retrieval, most classic approaches tend to represent different modalities of media in the same feature space. With the click data collected from the users' searching behavior, existing approaches take either one-to-one paired data (text-image pairs) or ranking examples (text-query-image and/or image-query-text ranking lists) as training examples, which do not make full use of the click data, particularly the implicit connections among the data objects. In this paper, we treat the click data as a large click graph, in which vertices are images/text queries and edges indicate the clicks between an image and a query. We consider learning a multimodal representation from the perspective of encoding the explicit/implicit relevance relationship between the vertices in the click graph. By minimizing both the truncated random walk loss as well as the distance between the learned representation of vertices and their corresponding deep neural network output, the proposed model which is named multimodal random walk neural network (MRW-NN) can be applied to not only learn robust representation of the existing multimodal data in the click graph, but also deal with the unseen queries and images to support cross-modal retrieval. We evaluate the latent representation learned by MRW-NN on a public large-scale click log data set Clickture and further show that MRW-NN achieves much better cross-modal retrieval performance on the unseen queries/images than the other state-of-the-art methods.

Multimodality Molecular Imaging-Guided Tumor Border Delineation and Photothermal Therapy Analysis Based on Graphene Oxide-Conjugated Gold Nanoparticles Chelated with Gd.

PubMed

Ma, Xibo; Jin, Yushen; Wang, Yi; Zhang, Shuai; Peng, Dong; Yang, Xin; Wei, Shoushui; Chai, Wei; Li, Xuejun; Tian, Jie

2018-01-01

Tumor cell complete extinction is a crucial measure to evaluate antitumor efficacy. The difficulties in defining tumor margins and finding satellite metastases are the reason for tumor recurrence. A synergistic method based on multimodality molecular imaging needs to be developed so as to achieve the complete extinction of the tumor cells. In this study, graphene oxide conjugated with gold nanostars and chelated with Gd through 1,4,7,10-tetraazacyclododecane-N,N',N,N'-tetraacetic acid (DOTA) (GO-AuNS-DOTA-Gd) were prepared to target HCC-LM3-fLuc cells and used for therapy. For subcutaneous tumor, multimodality molecular imaging including photoacoustic imaging (PAI) and magnetic resonance imaging (MRI) and the related processing techniques were used to monitor the pharmacokinetics process of GO-AuNS-DOTA-Gd in order to determine the optimal time for treatment. For orthotopic tumor, MRI was used to delineate the tumor location and margin in vivo before treatment. Then handheld photoacoustic imaging system was used to determine the tumor location during the surgery and guided the photothermal therapy. The experiment result based on orthotopic tumor demonstrated that this synergistic method could effectively reduce tumor residual and satellite metastases by 85.71% compared with the routine photothermal method without handheld PAI guidance. These results indicate that this multimodality molecular imaging-guided photothermal therapy method is promising with a good prospect in clinical application.

Novel multifunctional theranostic liposome drug delivery system: construction, characterization, and multimodality MR, near-infrared fluorescent, and nuclear imaging.

PubMed

Li, Shihong; Goins, Beth; Zhang, Lujun; Bao, Ande

2012-06-20

Liposomes are effective lipid nanoparticle drug delivery systems, which can also be functionalized with noninvasive multimodality imaging agents with each modality providing distinct information and having synergistic advantages in diagnosis, monitoring of disease treatment, and evaluation of liposomal drug pharmacokinetics. We designed and constructed a multifunctional theranostic liposomal drug delivery system, which integrated multimodality magnetic resonance (MR), near-infrared (NIR) fluorescent and nuclear imaging of liposomal drug delivery, and therapy monitoring and prediction. The premanufactured liposomes were composed of DSPC/cholesterol/Gd-DOTA-DSPE/DOTA-DSPE with the molar ratio of 39:35:25:1 and having ammonium sulfate/pH gradient. A lipidized NIR fluorescent tracer, IRDye-DSPE, was effectively postinserted into the premanufactured liposomes. Doxorubicin could be effectively postloaded into the multifunctional liposomes. The multifunctional doxorubicin-liposomes could also be stably radiolabeled with (99m)Tc or (64)Cu for single-photon emission computed tomography (SPECT) or positron emission tomography (PET) imaging, respectively. MR images displayed the high-resolution micro-intratumoral distribution of the liposomes in squamous cell carcinoma of head and neck (SCCHN) tumor xenografts in nude rats after intratumoral injection. NIR fluorescent, SPECT, and PET images also clearly showed either the high intratumoral retention or distribution of the multifunctional liposomes. This multifunctional drug carrying liposome system is promising for disease theranostics allowing noninvasive multimodality NIR fluorescent, MR, SPECT, and PET imaging of their in vivo behavior and capitalizing on the inherent advantages of each modality.

How acute and chronic alcohol consumption affects brain networks: insights from multimodal neuroimaging.

PubMed

Schulte, Tilman; Oberlin, Brandon G; Kareken, David A; Marinkovic, Ksenija; Müller-Oehring, Eva M; Meyerhoff, Dieter J; Tapert, Susan

2012-12-01

Multimodal imaging combining 2 or more techniques is becoming increasingly important because no single imaging approach has the capacity to elucidate all clinically relevant characteristics of a network. This review highlights recent advances in multimodal neuroimaging (i.e., combined use and interpretation of data collected through magnetic resonance imaging [MRI], functional MRI, diffusion tensor imaging, positron emission tomography, magnetoencephalography, MR perfusion, and MR spectroscopy methods) that leads to a more comprehensive understanding of how acute and chronic alcohol consumption affect neural networks underlying cognition, emotion, reward processing, and drinking behavior. Several innovative investigators have started utilizing multiple imaging approaches within the same individual to better understand how alcohol influences brain systems, both during intoxication and after years of chronic heavy use. Their findings can help identify mechanism-based therapeutic and pharmacological treatment options, and they may increase the efficacy and cost effectiveness of such treatments by predicting those at greatest risk for relapse. Copyright © 2012 by the Research Society on Alcoholism.

Dye-Enhanced Multimodal Confocal Imaging of Brain Cancers

NASA Astrophysics Data System (ADS)

Wirth, Dennis; Snuderl, Matija; Sheth, Sameer; Curry, William; Yaroslavsky, Anna

2011-04-01

Background and Significance: Accurate high resolution intraoperative detection of brain tumors may result in improved patient survival and better quality of life. The goal of this study was to evaluate dye enhanced multimodal confocal imaging for discriminating normal and cancerous brain tissue. Materials and Methods: Fresh thick brain specimens were obtained from the surgeries. Normal and cancer tissues were investigated. Samples were stained in methylene blue and imaged. Reflectance and fluorescence signals were excited at 658nm. Fluorescence emission and polarization were registered from 670 nm to 710 nm. The system provided lateral resolution of 0.6 μm and axial resolution of 7 μm. Normal and cancer specimens exhibited distinctively different characteristics. H&E histopathology was processed from each imaged sample. Results and Conclusions: The analysis of normal and cancerous tissues indicated clear differences in appearance in both the reflectance and fluorescence responses. These results confirm the feasibility of multimodal confocal imaging for intraoperative detection of small cancer nests and cells.

Combined multi-modal photoacoustic tomography, optical coherence tomography (OCT) and OCT angiography system with an articulated probe for in vivo human skin structure and vasculature imaging

PubMed Central

Liu, Mengyang; Chen, Zhe; Zabihian, Behrooz; Sinz, Christoph; Zhang, Edward; Beard, Paul C.; Ginner, Laurin; Hoover, Erich; Minneman, Micheal P.; Leitgeb, Rainer A.; Kittler, Harald; Drexler, Wolfgang

2016-01-01

Cutaneous blood flow accounts for approximately 5% of cardiac output in human and plays a key role in a number of a physiological and pathological processes. We show for the first time a multi-modal photoacoustic tomography (PAT), optical coherence tomography (OCT) and OCT angiography system with an articulated probe to extract human cutaneous vasculature in vivo in various skin regions. OCT angiography supplements the microvasculature which PAT alone is unable to provide. Co-registered volumes for vessel network is further embedded in the morphologic image provided by OCT. This multi-modal system is therefore demonstrated as a valuable tool for comprehensive non-invasive human skin vasculature and morphology imaging in vivo. PMID:27699106

Multimodal Nonlinear Optical Microscopy

PubMed Central

Yue, Shuhua; Slipchenko, Mikhail N.; Cheng, Ji-Xin

2013-01-01

Because each nonlinear optical (NLO) imaging modality is sensitive to specific molecules or structures, multimodal NLO imaging capitalizes the potential of NLO microscopy for studies of complex biological tissues. The coupling of multiphoton fluorescence, second harmonic generation, and coherent anti-Stokes Raman scattering (CARS) has allowed investigation of a broad range of biological questions concerning lipid metabolism, cancer development, cardiovascular disease, and skin biology. Moreover, recent research shows the great potential of using CARS microscope as a platform to develop more advanced NLO modalities such as electronic-resonance-enhanced four-wave mixing, stimulated Raman scattering, and pump-probe microscopy. This article reviews the various approaches developed for realization of multimodal NLO imaging as well as developments of new NLO modalities on a CARS microscope. Applications to various aspects of biological and biomedical research are discussed. PMID:24353747

Histological Underpinnings of Grey Matter Changes in Fibromyalgia Investigated Using Multimodal Brain Imaging.

PubMed

Pomares, Florence B; Funck, Thomas; Feier, Natasha A; Roy, Steven; Daigle-Martel, Alexandre; Ceko, Marta; Narayanan, Sridar; Araujo, David; Thiel, Alexander; Stikov, Nikola; Fitzcharles, Mary-Ann; Schweinhardt, Petra

2017-02-01

Chronic pain patients present with cortical gray matter alterations, observed with anatomical magnetic resonance (MR) imaging. Reduced regional gray matter volumes are often interpreted to reflect neurodegeneration, but studies investigating the cellular origin of gray matter changes are lacking. We used multimodal imaging to compare 26 postmenopausal women with fibromyalgia with 25 healthy controls (age range: 50-75 years) to test whether regional gray matter volume decreases in chronic pain are associated with compromised neuronal integrity. Regional gray matter decreases were largely explained by T1 relaxation times in gray matter, a surrogate measure of water content, and not to any substantial degree by GABA A receptor concentration, an indirect marker of neuronal integrity measured with [ 18 F] flumazenil PET. In addition, the MR spectroscopy marker of neuronal viability, N-acetylaspartate, did not differ between patients and controls. These findings suggest that decreased gray matter volumes are not explained by compromised neuronal integrity. Alternatively, a decrease in neuronal matter could be compensated for by an upregulation of GABA A receptors. The relation between regional gray matter and T1 relaxation times suggests decreased tissue water content underlying regional gray matter decreases. In contrast, regional gray matter increases were explained by GABA A receptor concentration in addition to T1 relaxation times, indicating perhaps increased neuronal matter or GABA A receptor upregulation and inflammatory edema. By providing information on the histological origins of cerebral gray matter alterations in fibromyalgia, this study advances the understanding of the neurobiology of chronic widespread pain. Regional gray matter alterations in chronic pain, as detected with voxel-based morphometry of anatomical magnetic resonance images, are commonly interpreted to reflect neurodegeneration, but this assumption has not been tested. We found decreased gray matter in fibromyalgia to be associated with T1 relaxation times, a surrogate marker of water content, but not with GABA A receptor concentration, a surrogate of neuronal integrity. In contrast, regional gray matter increases were partly explained by GABA A receptor concentration, indicating some form of neuronal plasticity. The study emphasizes that voxel-based morphometry is an exploratory measure, demonstrating the need to investigate the histological origin of gray matter alterations for every distinct clinical entity, and advances the understanding of the neurobiology of chronic (widespread) pain. Copyright © 2017 the authors 0270-6474/17/371091-12$15.00/0.

Fluorine-18-labeled Gd3+/Yb3+/Er3+ co-doped NaYF4 nanophosphors for multimodality PET/MR/UCL imaging.

PubMed

Zhou, Jing; Yu, Mengxiao; Sun, Yun; Zhang, Xianzhong; Zhu, Xingjun; Wu, Zhanhong; Wu, Dongmei; Li, Fuyou

2011-02-01

Molecular imaging modalities provide a wealth of information that is highly complementary and rarely redundant. To combine the advantages of molecular imaging techniques, (18)F-labeled Gd(3+)/Yb(3+)/Er(3+) co-doped NaYF(4) nanophosphors (NPs) simultaneously possessing with radioactivity, magnetic, and upconversion luminescent properties have been fabricated for multimodality positron emission tomography (PET), magnetic resonance imaging (MRI), and laser scanning upconversion luminescence (UCL) imaging. Hydrophilic citrate-capped NaY(0.2)Gd(0.6)Yb(0.18)Er(0.02)F(4) nanophosphors (cit-NPs) were obtained from hydrophobic oleic acid (OA)-coated nanoparticles (OA-NPs) through a process of ligand exchange of OA with citrate, and were found to be monodisperse with an average size of 22 × 19 nm. The obtained hexagonal cit-NPs show intense UCL emission in the visible region and paramagnetic longitudinal relaxivity (r(1) = 0.405 s(-1)·(mM)(-1)). Through a facile inorganic reaction based on the strong binding between Y(3+) and F(-), (18)F-labeled NPs have been fabricated in high yield. The use of cit-NPs as a multimodal probe has been further explored for T(1)-weighted MR and PET imaging in vivo and UCL imaging of living cells and tissue slides. The results indicate that (18)F-labeled NaY(0.2)Gd(0.6)Yb(0.18)Er(0.02) is a potential candidate as a multimodal nanoprobe for ultra-sensitive molecular imaging from the cellular scale to whole-body evaluation. Copyright © 2010 Elsevier Ltd. All rights reserved.

Multimodal nonlinear microscopy of biopsy specimen: towards intraoperative diagnostics (Conference Presentation)

NASA Astrophysics Data System (ADS)

Schmitt, Michael; Heuke, Sandro; Meyer, Tobias; Chernavskaia, Olga; Bocklitz, Thomas W.; Popp, Juergen

2016-03-01

The realization of label-free molecule specific imaging of morphology and chemical composition of tissue at subcellular spatial resolution in real time is crucial for many envisioned applications in medicine, e.g., precise surgical guidance and non-invasive histopathologic examination of tissue. Thus, new approaches for a fast and reliable in vivo and near in vivo (ex corpore in vivo) tissue characterization to supplement routine pathological diagnostics is needed. Spectroscopic imaging approaches are particularly important since they have the potential to provide a pathologist with adequate support in the form of clinically-relevant information under both ex vivo and in vivo conditions. In this contribution it is demonstrated, that multimodal nonlinear microscopy combining coherent anti-Stokes Raman scattering (CARS), two photon excited fluorescence (TPEF) and second harmonic generation (SHG) enables the detection of characteristic structures and the accompanying molecular changes of widespread diseases, particularly of cancer and atherosclerosis. The detailed images enable an objective evaluation of the tissue samples for an early diagnosis of the disease status. Increasing the spectral resolution and analyzing CARS images at multiple Raman resonances improves the chemical specificity. To facilitate handling and interpretation of the image data characteristic properties can be automatically extracted by advanced image processing algorithms, e.g., for tissue classification. Overall, the presented examples show the great potential of multimodal imaging to augment standard intraoperative clinical assessment with functional multimodal CARS/SHG/TPEF images to highlight functional activity and tumor boundaries. It ensures fast, label-free and non-invasive intraoperative tissue classification paving the way towards in vivo optical pathology.

Au Nanocage Functionalized with Ultra-small Fe3O4 Nanoparticles for Targeting T1-T2Dual MRI and CT Imaging of Tumor

NASA Astrophysics Data System (ADS)

Wang, Guannan; Gao, Wei; Zhang, Xuanjun; Mei, Xifan

2016-06-01

Diagnostic approaches based on multimodal imaging of clinical noninvasive imaging (eg. MRI/CT scanner) are highly developed in recent years for accurate selection of the therapeutic regimens in critical diseases. Therefore, it is highly demanded in the development of appropriate all-in-one multimodal contrast agents (MCAs) for the MRI/CT multimodal imaging. Here a novel ideal MCAs (F-AuNC@Fe3O4) were engineered by assemble Au nanocages (Au NC) and ultra-small iron oxide nanoparticles (Fe3O4) for simultaneous T1-T2dual MRI and CT contrast imaging. In this system, the Au nanocages offer facile thiol modification and strong X-ray attenuation property for CT imaging. The ultra-small Fe3O4 nanoparticles, as excellent contrast agent, is able to provide great enhanced signal of T1- and T2-weighted MRI (r1 = 6.263 mM-1 s-1, r2 = 28.117 mM-1 s-1) due to their ultra-refined size. After functionalization, the present MCAs nanoparticles exhibited small average size, low aggregation and excellent biocompatible. In vitro and In vivo studies revealed that the MCAs show long-term circulation time, renal clearance properties and outstanding capability of selective accumulation in tumor tissues for simultaneous CT imaging and T1- and T2-weighted MRI. Taken together, these results show that as-prepared MCAs are excellent candidates as MRI/CT multimodal imaging contrast agents.

A prototype hand-held tri-modal instrument for in vivo ultrasound, photoacoustic, and fluorescence imaging

NASA Astrophysics Data System (ADS)

Kang, Jeeun; Chang, Jin Ho; Wilson, Brian C.; Veilleux, Israel; Bai, Yanhui; DaCosta, Ralph; Kim, Kang; Ha, Seunghan; Lee, Jong Gun; Kim, Jeong Seok; Lee, Sang-Goo; Kim, Sun Mi; Lee, Hak Jong; Ahn, Young Bok; Han, Seunghee; Yoo, Yangmo; Song, Tai-Kyong

2015-03-01

Multi-modality imaging is beneficial for both preclinical and clinical applications as it enables complementary information from each modality to be obtained in a single procedure. In this paper, we report the design, fabrication, and testing of a novel tri-modal in vivo imaging system to exploit molecular/functional information from fluorescence (FL) and photoacoustic (PA) imaging as well as anatomical information from ultrasound (US) imaging. The same ultrasound transducer was used for both US and PA imaging, bringing the pulsed laser light into a compact probe by fiberoptic bundles. The FL subsystem is independent of the acoustic components but the front end that delivers and collects the light is physically integrated into the same probe. The tri-modal imaging system was implemented to provide each modality image in real time as well as co-registration of the images. The performance of the system was evaluated through phantom and in vivo animal experiments. The results demonstrate that combining the modalities does not significantly compromise the performance of each of the separate US, PA, and FL imaging techniques, while enabling multi-modality registration. The potential applications of this novel approach to multi-modality imaging range from preclinical research to clinical diagnosis, especially in detection/localization and surgical guidance of accessible solid tumors.

Showing or Telling a Story: A Comparative Study of Public Education Texts in Multimodality and Monomodality

ERIC Educational Resources Information Center

Wang, Kelu

2013-01-01

Multimodal texts that combine words and images produce meaning in a different way from monomodal texts that rely on words. They differ not only in representing the subject matter, but also constructing relationships between text producers and text receivers. This article uses two multimodal texts and one monomodal written text as samples, which…

Label-aligned Multi-task Feature Learning for Multimodal Classification of Alzheimer’s Disease and Mild Cognitive Impairment

PubMed Central

Zu, Chen; Jie, Biao; Liu, Mingxia; Chen, Songcan

2015-01-01

Multimodal classification methods using different modalities of imaging and non-imaging data have recently shown great advantages over traditional single-modality-based ones for diagnosis and prognosis of Alzheimer’s disease (AD), as well as its prodromal stage, i.e., mild cognitive impairment (MCI). However, to the best of our knowledge, most existing methods focus on mining the relationship across multiple modalities of the same subjects, while ignoring the potentially useful relationship across different subjects. Accordingly, in this paper, we propose a novel learning method for multimodal classification of AD/MCI, by fully exploring the relationships across both modalities and subjects. Specifically, our proposed method includes two subsequent components, i.e., label-aligned multi-task feature selection and multimodal classification. In the first step, the feature selection learning from multiple modalities are treated as different learning tasks and a group sparsity regularizer is imposed to jointly select a subset of relevant features. Furthermore, to utilize the discriminative information among labeled subjects, a new label-aligned regularization term is added into the objective function of standard multi-task feature selection, where label-alignment means that all multi-modality subjects with the same class labels should be closer in the new feature-reduced space. In the second step, a multi-kernel support vector machine (SVM) is adopted to fuse the selected features from multi-modality data for final classification. To validate our method, we perform experiments on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database using baseline MRI and FDG-PET imaging data. The experimental results demonstrate that our proposed method achieves better classification performance compared with several state-of-the-art methods for multimodal classification of AD/MCI. PMID:26572145

GMars-T Enabling Multimodal Subdiffraction Structural and Functional Fluorescence Imaging in Live Cells.

PubMed

Wang, Sheng; Chen, Xuanze; Chang, Lei; Ding, Miao; Xue, Ruiying; Duan, Haifeng; Sun, Yujie

2018-06-05

Fluorescent probes with multimodal and multilevel imaging capabilities are highly valuable as imaging with such probes not only can obtain new layers of information but also enable cross-validation of results under different experimental conditions. In recent years, the development of genetically encoded reversibly photoswitchable fluorescent proteins (RSFPs) has greatly promoted the application of various kinds of live-cell nanoscopy approaches, including reversible saturable optical fluorescence transitions (RESOLFT) and stochastic optical fluctuation imaging (SOFI). However, these two classes of live-cell nanoscopy approaches require different optical characteristics of specific RSFPs. In this work, we developed GMars-T, a monomeric bright green RSFP which can satisfy both RESOLFT and photochromic SOFI (pcSOFI) imaging in live cells. We further generated biosensor based on bimolecular fluorescence complementation (BiFC) of GMars-T which offers high specificity and sensitivity in detecting and visualizing various protein-protein interactions (PPIs) in different subcellular compartments under physiological conditions (e.g., 37 °C) in live mammalian cells. Thus, the newly developed GMars-T can serve as both structural imaging probe with multimodal super-resolution imaging capability and functional imaging probe for reporting PPIs with high specificity and sensitivity based on its derived biosensor.

Automatic segmentation of multimodal brain tumor images based on classification of super-voxels.

PubMed

Kadkhodaei, M; Samavi, S; Karimi, N; Mohaghegh, H; Soroushmehr, S M R; Ward, K; All, A; Najarian, K

2016-08-01

Despite the rapid growth in brain tumor segmentation approaches, there are still many challenges in this field. Automatic segmentation of brain images has a critical role in decreasing the burden of manual labeling and increasing robustness of brain tumor diagnosis. We consider segmentation of glioma tumors, which have a wide variation in size, shape and appearance properties. In this paper images are enhanced and normalized to same scale in a preprocessing step. The enhanced images are then segmented based on their intensities using 3D super-voxels. Usually in images a tumor region can be regarded as a salient object. Inspired by this observation, we propose a new feature which uses a saliency detection algorithm. An edge-aware filtering technique is employed to align edges of the original image to the saliency map which enhances the boundaries of the tumor. Then, for classification of tumors in brain images, a set of robust texture features are extracted from super-voxels. Experimental results indicate that our proposed method outperforms a comparable state-of-the-art algorithm in term of dice score.

Introduction to clinical and laboratory (small-animal) image registration and fusion.

PubMed

Zanzonico, Pat B; Nehmeh, Sadek A

2006-01-01

Imaging has long been a vital component of clinical medicine and, increasingly, of biomedical research in small-animals. Clinical and laboratory imaging modalities can be divided into two general categories, structural (or anatomical) and functional (or physiological). The latter, in particular, has spawned what has come to be known as "molecular imaging". Image registration and fusion have rapidly emerged as invaluable components of both clinical and small-animal imaging and has lead to the development and marketing of a variety of multi-modality, e.g. PET-CT, devices which provide registered and fused three-dimensional image sets. This paper briefly reviews the basics of image registration and fusion and available clinical and small-animal multi-modality instrumentation.

Multi-Modal Curriculum Learning for Semi-Supervised Image Classification.

PubMed

Gong, Chen; Tao, Dacheng; Maybank, Stephen J; Liu, Wei; Kang, Guoliang; Yang, Jie

2016-07-01

Semi-supervised image classification aims to classify a large quantity of unlabeled images by typically harnessing scarce labeled images. Existing semi-supervised methods often suffer from inadequate classification accuracy when encountering difficult yet critical images, such as outliers, because they treat all unlabeled images equally and conduct classifications in an imperfectly ordered sequence. In this paper, we employ the curriculum learning methodology by investigating the difficulty of classifying every unlabeled image. The reliability and the discriminability of these unlabeled images are particularly investigated for evaluating their difficulty. As a result, an optimized image sequence is generated during the iterative propagations, and the unlabeled images are logically classified from simple to difficult. Furthermore, since images are usually characterized by multiple visual feature descriptors, we associate each kind of features with a teacher, and design a multi-modal curriculum learning (MMCL) strategy to integrate the information from different feature modalities. In each propagation, each teacher analyzes the difficulties of the currently unlabeled images from its own modality viewpoint. A consensus is subsequently reached among all the teachers, determining the currently simplest images (i.e., a curriculum), which are to be reliably classified by the multi-modal learner. This well-organized propagation process leveraging multiple teachers and one learner enables our MMCL to outperform five state-of-the-art methods on eight popular image data sets.

Multi-Modal Performance Measures in Oregon: Developing a Transportation Cost Index Based Upon Multi-Modal Network and Land Use Information

DOT National Transportation Integrated Search

2016-02-01

Transportation Cost Index is a performance measure for transportation and land use systems originally proposed and piloted by Reiff and Gregor (2005). It fills important niches of existing similar measures in term of policy areas covered and type of ...

Deep features for efficient multi-biometric recognition with face and ear images

NASA Astrophysics Data System (ADS)

Omara, Ibrahim; Xiao, Gang; Amrani, Moussa; Yan, Zifei; Zuo, Wangmeng

2017-07-01

Recently, multimodal biometric systems have received considerable research interest in many applications especially in the fields of security. Multimodal systems can increase the resistance to spoof attacks, provide more details and flexibility, and lead to better performance and lower error rate. In this paper, we present a multimodal biometric system based on face and ear, and propose how to exploit the extracted deep features from Convolutional Neural Networks (CNNs) on the face and ear images to introduce more powerful discriminative features and robust representation ability for them. First, the deep features for face and ear images are extracted based on VGG-M Net. Second, the extracted deep features are fused by using a traditional concatenation and a Discriminant Correlation Analysis (DCA) algorithm. Third, multiclass support vector machine is adopted for matching and classification. The experimental results show that the proposed multimodal system based on deep features is efficient and achieves a promising recognition rate up to 100 % by using face and ear. In addition, the results indicate that the fusion based on DCA is superior to traditional fusion.

Multimodal optical imaging system for in vivo investigation of cerebral oxygen delivery and energy metabolism

PubMed Central

Yaseen, Mohammad A.; Srinivasan, Vivek J.; Gorczynska, Iwona; Fujimoto, James G.; Boas, David A.; Sakadžić, Sava

2015-01-01

Improving our understanding of brain function requires novel tools to observe multiple physiological parameters with high resolution in vivo. We have developed a multimodal imaging system for investigating multiple facets of cerebral blood flow and metabolism in small animals. The system was custom designed and features multiple optical imaging capabilities, including 2-photon and confocal lifetime microscopy, optical coherence tomography, laser speckle imaging, and optical intrinsic signal imaging. Here, we provide details of the system’s design and present in vivo observations of multiple metrics of cerebral oxygen delivery and energy metabolism, including oxygen partial pressure, microvascular blood flow, and NADH autofluorescence. PMID:26713212

Computational method for multi-modal microscopy based on transport of intensity equation

NASA Astrophysics Data System (ADS)

Li, Jiaji; Chen, Qian; Sun, Jiasong; Zhang, Jialin; Zuo, Chao

2017-02-01

In this paper, we develop the requisite theory to describe a hybrid virtual-physical multi-modal imaging system which yields quantitative phase, Zernike phase contrast, differential interference contrast (DIC), and light field moment imaging simultaneously based on transport of intensity equation(TIE). We then give the experimental demonstration of these ideas by time-lapse imaging of live HeLa cell mitosis. Experimental results verify that a tunable lens based TIE system, combined with the appropriate post-processing algorithm, can achieve a variety of promising imaging modalities in parallel with the quantitative phase images for the dynamic study of cellular processes.

Multimodality Molecular Imaging of Cardiac Cell Transplantation: Part I. Reporter Gene Design, Characterization, and Optical in Vivo Imaging of Bone Marrow Stromal Cells after Myocardial Infarction

PubMed Central

Parashurama, Natesh; Ahn, Byeong-Cheol; Ziv, Keren; Ito, Ken; Paulmurugan, Ramasamy; Willmann, Jürgen K.; Chung, Jaehoon; Ikeno, Fumiaki; Swanson, Julia C.; Merk, Denis R.; Lyons, Jennifer K.; Yerushalmi, David; Teramoto, Tomohiko; Kosuge, Hisanori; Dao, Catherine N.; Ray, Pritha; Patel, Manishkumar; Chang, Ya-fang; Mahmoudi, Morteza; Cohen, Jeff Eric; Goldstone, Andrew Brooks; Habte, Frezghi; Bhaumik, Srabani; Yaghoubi, Shahriar; Robbins, Robert C.; Dash, Rajesh; Yang, Phillip C.; Brinton, Todd J.; Yock, Paul G.; McConnell, Michael V.

2016-01-01

Purpose To use multimodality reporter-gene imaging to assess the serial survival of marrow stromal cells (MSC) after therapy for myocardial infarction (MI) and to determine if the requisite preclinical imaging end point was met prior to a follow-up large-animal MSC imaging study. Materials and Methods Animal studies were approved by the Institutional Administrative Panel on Laboratory Animal Care. Mice (n = 19) that had experienced MI were injected with bone marrow–derived MSC that expressed a multimodality triple fusion (TF) reporter gene. The TF reporter gene (fluc2-egfp-sr39ttk) consisted of a human promoter, ubiquitin, driving firefly luciferase 2 (fluc2), enhanced green fluorescent protein (egfp), and the sr39tk positron emission tomography reporter gene. Serial bioluminescence imaging of MSC-TF and ex vivo luciferase assays were performed. Correlations were analyzed with the Pearson product-moment correlation, and serial imaging results were analyzed with a mixed-effects regression model. Results Analysis of the MSC-TF after cardiac cell therapy showed significantly lower signal on days 8 and 14 than on day 2 (P = .011 and P = .001, respectively). MSC-TF with MI demonstrated significantly higher signal than MSC-TF without MI at days 4, 8, and 14 (P = .016). Ex vivo luciferase activity assay confirmed the presence of MSC-TF on days 8 and 14 after MI. Conclusion Multimodality reporter-gene imaging was successfully used to assess serial MSC survival after therapy for MI, and it was determined that the requisite preclinical imaging end point, 14 days of MSC survival, was met prior to a follow-up large-animal MSC study. © RSNA, 2016 Online supplemental material is available for this article. PMID:27308957

Drusen Characterization with Multimodal Imaging

PubMed Central

Spaide, Richard F.; Curcio, Christine A.

2010-01-01

Summary Multimodal imaging findings and histological demonstration of soft drusen, cuticular drusen, and subretinal drusenoid deposits provided information used to develop a model explaining their imaging characteristics. Purpose To characterize the known appearance of cuticular drusen, subretinal drusenoid deposits (reticular pseudodrusen), and soft drusen as revealed by multimodal fundus imaging; to create an explanatory model that accounts for these observations. Methods Reported color, fluorescein angiographic, autofluorescence, and spectral domain optical coherence tomography (SD-OCT) images of patients with cuticular drusen, soft drusen, and subretinal drusenoid deposits were reviewed, as were actual images from affected eyes. Representative histological sections were examined. The geometry, location, and imaging characteristics of these lesions were evaluated. A hypothesis based on the Beer-Lambert Law of light absorption was generated to fit these observations. Results Cuticular drusen appear as numerous uniform round yellow-white punctate accumulations under the retinal pigment epithelium (RPE). Soft drusen are larger yellow-white dome-shaped mounds of deposit under the RPE. Subretinal drusenoid deposits are polymorphous light-grey interconnected accumulations above the RPE. Based on the model, both cuticular and soft drusen appear yellow due to the removal of shorter wavelength light by a double pass through the RPE. Subretinal drusenoid deposits, which are located on the RPE, are not subjected to short wavelength attenuation and therefore are more prominent when viewed with blue light. The location and morphology of extracellular material in relationship to the RPE, and associated changes to RPE morphology and pigmentation, appeared to be primary determinants of druse appearance in different imaging modalities. Conclusion Although cuticular drusen, subretinal drusenoid deposits, and soft drusen are composed of common components, they are distinguishable by multimodal imaging due to differences in location, morphology, and optical filtering effects by drusenoid material and the RPE. PMID:20924263

Tunable X-ray speckle-based phase-contrast and dark-field imaging using the unified modulated pattern analysis approach

NASA Astrophysics Data System (ADS)

Zdora, M.-C.; Thibault, P.; Deyhle, H.; Vila-Comamala, J.; Rau, C.; Zanette, I.

2018-05-01

X-ray phase-contrast and dark-field imaging provides valuable, complementary information about the specimen under study. Among the multimodal X-ray imaging methods, X-ray grating interferometry and speckle-based imaging have drawn particular attention, which, however, in their common implementations incur certain limitations that can restrict their range of applications. Recently, the unified modulated pattern analysis (UMPA) approach was proposed to overcome these limitations and combine grating- and speckle-based imaging in a single approach. Here, we demonstrate the multimodal imaging capabilities of UMPA and highlight its tunable character regarding spatial resolution, signal sensitivity and scan time by using different reconstruction parameters.

Multimodality imaging of ovarian cystic lesions: Review with an imaging based algorithmic approach

PubMed Central

Wasnik, Ashish P; Menias, Christine O; Platt, Joel F; Lalchandani, Usha R; Bedi, Deepak G; Elsayes, Khaled M

2013-01-01

Ovarian cystic masses include a spectrum of benign, borderline and high grade malignant neoplasms. Imaging plays a crucial role in characterization and pretreatment planning of incidentally detected or suspected adnexal masses, as diagnosis of ovarian malignancy at an early stage is correlated with a better prognosis. Knowledge of differential diagnosis, imaging features, management trends and an algorithmic approach of such lesions is important for optimal clinical management. This article illustrates a multi-modality approach in the diagnosis of a spectrum of ovarian cystic masses and also proposes an algorithmic approach for the diagnosis of these lesions. PMID:23671748

A systematic, multimodality approach to emergency elbow imaging.

PubMed

Singer, Adam D; Hanna, Tarek; Jose, Jean; Datir, Abhijit

2016-01-01

The elbow is a complex synovial hinge joint that is frequently involved in both athletic and nonathletic injuries. A thorough understanding of the normal anatomy and various injury patterns is essential when utilizing diagnostic imaging to identify damaged structures and to assist in surgical planning. In this review, the elbow anatomy will be scrutinized in a systematic approach. This will be followed by a comprehensive presentation of elbow injuries that are commonly seen in the emergency department accompanied by multimodality imaging findings. A short discussion regarding pitfalls in elbow imaging is also included. Copyright © 2015 Elsevier Inc. All rights reserved.

Radiomic biomarkers from PET/CT multi-modality fusion images for the prediction of immunotherapy response in advanced non-small cell lung cancer patients

NASA Astrophysics Data System (ADS)

Mu, Wei; Qi, Jin; Lu, Hong; Schabath, Matthew; Balagurunathan, Yoganand; Tunali, Ilke; Gillies, Robert James

2018-02-01

Purpose: Investigate the ability of using complementary information provided by the fusion of PET/CT images to predict immunotherapy response in non-small cell lung cancer (NSCLC) patients. Materials and methods: We collected 64 patients diagnosed with primary NSCLC treated with anti PD-1 checkpoint blockade. Using PET/CT images, fused images were created following multiple methodologies, resulting in up to 7 different images for the tumor region. Quantitative image features were extracted from the primary image (PET/CT) and the fused images, which included 195 from primary images and 1235 features from the fusion images. Three clinical characteristics were also analyzed. We then used support vector machine (SVM) classification models to identify discriminant features that predict immunotherapy response at baseline. Results: A SVM built with 87 fusion features and 13 primary PET/CT features on validation dataset had an accuracy and area under the ROC curve (AUROC) of 87.5% and 0.82, respectively, compared to a model built with 113 original PET/CT features on validation dataset 78.12% and 0.68. Conclusion: The fusion features shows better ability to predict immunotherapy response prediction compared to individual image features.

Multimode-Optical-Fiber Imaging Probe

NASA Technical Reports Server (NTRS)

Jackson, Deborah

2000-01-01

Currently, endoscopic surgery uses single-mode fiber-bundles to obtain in vivo image information inside orifices of the body. This limits their use to the larger natural bodily orifices and to surgical procedures where there is plenty of room for manipulation. The knee joint, for example can be easily viewed with a fiber optic viewer, but joints in the finger cannot. However, there are a host of smaller orifices where fiber endoscopy would play an important role if a cost effective fiber probe were developed with small enough dimensions (< 250 microns). Examples of beneficiaries of micro-endoscopes are the treatment of the Eustatian tube of the middle ear, the breast ducts, tear ducts, coronary arteries, fallopian tubes, as well as the treatment of salivary duct parotid disease, and the neuro endoscopy of the ventricles and spinal canal. To solve this problem, this work describes an approach for recovering images from. tightly confined spaces using multimode fibers and analytically demonstrates that the concept is sound. The proof of concept draws upon earlier works that concentrated on image recovery after two-way transmission through a multimode fiber as well as work that demonstrated the recovery of images after one-way transmission through a multimode fiber. Both relied on generating a phase conjugated wavefront which was predistorted with the characteristics of the fiber. The described approach also relies on generating a phase conjugated wavefront, but utilizes two fibers to capture the image at some intermediate point (accessible by the fibers, but which is otherwise visually unaccessible).

Multimode-Optical-Fiber Imaging Probe

NASA Technical Reports Server (NTRS)

Jackson, Deborah

1999-01-01

Currently, endoscopic surgery uses single-mode fiber-bundles to obtain in vivo image information inside the orifices of the body. This limits their use to the larger natural orifices and to surgical procedures where there is plenty of room for manipulation. The knee joint, for example, can be easily viewed with a fiber optic viewer, but joints in the finger cannot. However, there are a host of smaller orifices where fiber endoscopy would play an important role if a cost effective fiber probe were developed with small enough dimensions (less than or equal to 250 microns). Examples of beneficiaries of micro-endoscopes are the treatment of the Eustatian tube of the middle ear, the breast ducts, tear ducts, coronary arteries, fallopian tubes, as well as the treatment of salivary duct parotid disease, and the neuro endoscopy of the ventricles and spinal canal. This work describes an approach for recovering images from tightly confined spaces using multimode. The concept draws upon earlier works that concentrated on image recovery after two-way transmission through a multimode fiber as well as work that demonstrated the recovery of images after one-way transmission through a multimode fiber. Both relied on generating a phase conjugated wavefront, which was predistorted with the characteristics of the fiber. The approach described here also relies on generating a phase conjugated wavefront, but utilizes two fibers to capture the image at some intermediate point (accessible by the fibers, but which is otherwise visually inaccessible).

Patient-tailored multimodal neuroimaging, visualization and quantification of human intra-cerebral hemorrhage

NASA Astrophysics Data System (ADS)

Goh, Sheng-Yang M.; Irimia, Andrei; Vespa, Paul M.; Van Horn, John D.

2016-03-01

In traumatic brain injury (TBI) and intracerebral hemorrhage (ICH), the heterogeneity of lesion sizes and types necessitates a variety of imaging modalities to acquire a comprehensive perspective on injury extent. Although it is advantageous to combine imaging modalities and to leverage their complementary benefits, there are difficulties in integrating information across imaging types. Thus, it is important that efforts be dedicated to the creation and sustained refinement of resources for multimodal data integration. Here, we propose a novel approach to the integration of neuroimaging data acquired from human patients with TBI/ICH using various modalities; we also demonstrate the integrated use of multimodal magnetic resonance imaging (MRI) and diffusion tensor imaging (DTI) data for TBI analysis based on both visual observations and quantitative metrics. 3D models of healthy-appearing tissues and TBIrelated pathology are generated, both of which are derived from multimodal imaging data. MRI volumes acquired using FLAIR, SWI, and T2 GRE are used to segment pathology. Healthy tissues are segmented using user-supervised tools, and results are visualized using a novel graphical approach called a `connectogram', where brain connectivity information is depicted within a circle of radially aligned elements. Inter-region connectivity and its strength are represented by links of variable opacities drawn between regions, where opacity reflects the percentage longitudinal change in brain connectivity density. Our method for integrating, analyzing and visualizing structural brain changes due to TBI and ICH can promote knowledge extraction and enhance the understanding of mechanisms underlying recovery.

MMX-I: A data-processing software for multi-modal X-ray imaging and tomography

NASA Astrophysics Data System (ADS)

Bergamaschi, A.; Medjoubi, K.; Messaoudi, C.; Marco, S.; Somogyi, A.

2017-06-01

Scanning hard X-ray imaging allows simultaneous acquisition of multimodal information, including X-ray fluorescence, absorption, phase and dark-field contrasts, providing structural and chemical details of the samples. Combining these scanning techniques with the infrastructure developed for fast data acquisition at Synchrotron Soleil permits to perform multimodal imaging and tomography during routine user experiments at the Nanoscopium beamline. A main challenge of such imaging techniques is the online processing and analysis of the generated very large volume (several hundreds of Giga Bytes) multimodal data-sets. This is especially important for the wide user community foreseen at the user oriented Nanoscopium beamline (e.g. from the fields of Biology, Life Sciences, Geology, Geobiology), having no experience in such data-handling. MMX-I is a new multi-platform open-source freeware for the processing and reconstruction of scanning multi-technique X-ray imaging and tomographic datasets. The MMX-I project aims to offer, both expert users and beginners, the possibility of processing and analysing raw data, either on-site or off-site. Therefore we have developed a multi-platform (Mac, Windows and Linux 64bit) data processing tool, which is easy to install, comprehensive, intuitive, extendable and user-friendly. MMX-I is now routinely used by the Nanoscopium user community and has demonstrated its performance in treating big data.

Image-guided thoracic surgery in the hybrid operation room.

PubMed

Ujiie, Hideki; Effat, Andrew; Yasufuku, Kazuhiro

2017-01-01

There has been an increase in the use of image-guided technology to facilitate minimally invasive therapy. The next generation of minimally invasive therapy is focused on advancement and translation of novel image-guided technologies in therapeutic interventions, including surgery, interventional pulmonology, radiation therapy, and interventional laser therapy. To establish the efficacy of different minimally invasive therapies, we have developed a hybrid operating room, known as the guided therapeutics operating room (GTx OR) at the Toronto General Hospital. The GTx OR is equipped with multi-modality image-guidance systems, which features a dual source-dual energy computed tomography (CT) scanner, a robotic cone-beam CT (CBCT)/fluoroscopy, high-performance endobronchial ultrasound system, endoscopic surgery system, near-infrared (NIR) fluorescence imaging system, and navigation tracking systems. The novel multimodality image-guidance systems allow physicians to quickly, and accurately image patients while they are on the operating table. This yield improved outcomes since physicians are able to use image guidance during their procedures, and carry out innovative multi-modality therapeutics. Multiple preclinical translational studies pertaining to innovative minimally invasive technology is being developed in our guided therapeutics laboratory (GTx Lab). The GTx Lab is equipped with similar technology, and multimodality image-guidance systems as the GTx OR, and acts as an appropriate platform for translation of research into human clinical trials. Through the GTx Lab, we are able to perform basic research, such as the development of image-guided technologies, preclinical model testing, as well as preclinical imaging, and then translate that research into the GTx OR. This OR allows for the utilization of new technologies in cancer therapy, including molecular imaging, and other innovative imaging modalities, and therefore enables a better quality of life for patients, both during and after the procedure. In this article, we describe capabilities of the GTx systems, and discuss the first-in-human technologies used, and evaluated in GTx OR.

WE-H-206-02: Recent Advances in Multi-Modality Molecular Imaging of Small Animals

DOE Office of Scientific and Technical Information (OSTI.GOV)

Tsui, B.

Lihong V. Wang: Photoacoustic tomography (PAT), combining non-ionizing optical and ultrasonic waves via the photoacoustic effect, provides in vivo multiscale functional, metabolic, and molecular imaging. Broad applications include imaging of the breast, brain, skin, esophagus, colon, vascular system, and lymphatic system in humans or animals. Light offers rich contrast but does not penetrate biological tissue in straight paths as x-rays do. Consequently, high-resolution pure optical imaging (e.g., confocal microscopy, two-photon microscopy, and optical coherence tomography) is limited to penetration within the optical diffusion limit (∼1 mm in the skin). Ultrasonic imaging, on the contrary, provides fine spatial resolution but suffersmore » from both poor contrast in early-stage tumors and strong speckle artifacts. In PAT, pulsed laser light penetrates tissue and generates a small but rapid temperature rise, which induces emission of ultrasonic waves due to thermoelastic expansion. The ultrasonic waves, orders of magnitude less scattering than optical waves, are then detected to form high-resolution images of optical absorption at depths up to 7 cm, conquering the optical diffusion limit. PAT is the only modality capable of imaging across the length scales of organelles, cells, tissues, and organs (up to whole-body small animals) with consistent contrast. This rapidly growing technology promises to enable multiscale biological research and accelerate translation from microscopic laboratory discoveries to macroscopic clinical practice. PAT may also hold the key to label-free early detection of cancer by in vivo quantification of hypermetabolism, the quintessential hallmark of malignancy. Learning Objectives: To understand the contrast mechanism of PAT To understand the multiscale applications of PAT Benjamin M. W. Tsui: Multi-modality molecular imaging instrumentation and techniques have been major developments in small animal imaging that has contributed significantly to biomedical research during the past decade. The initial development was an extension of clinical PET/CT and SPECT/CT from human to small animals and combine the unique functional information obtained from PET and SPECT with anatomical information provided by the CT in registered multi-modality images. The requirements to image a mouse whose size is an order of magnitude smaller than that of a human have spurred advances in new radiation detector technologies, novel imaging system designs and special image reconstruction and processing techniques. Examples are new detector materials and designs with high intrinsic resolution, multi-pinhole (MPH) collimator design for much improved resolution and detection efficiency compared to the conventional collimator designs in SPECT, 3D high-resolution and artifact-free MPH and sparse-view image reconstruction techniques, and iterative image reconstruction methods with system response modeling for resolution recovery and image noise reduction for much improved image quality. The spatial resolution of PET and SPECT has improved from ∼6–12 mm to ∼1 mm a few years ago to sub-millimeter today. A recent commercial small animal SPECT system has achieved a resolution of ∼0.25 mm which surpasses that of a state-of-art PET system whose resolution is limited by the positron range. More recently, multimodality SA PET/MRI and SPECT/MRI systems have been developed in research laboratories. Also, multi-modality SA imaging systems that include other imaging modalities such as optical and ultrasound are being actively pursued. In this presentation, we will provide a review of the development, recent advances and future outlook of multi-modality molecular imaging of small animals. Learning Objectives: To learn about the two major multi-modality molecular imaging techniques of small animals. To learn about the spatial resolution achievable by the molecular imaging systems for small animal today. To learn about the new multi-modality imaging instrumentation and techniques that are being developed. Sang Hyun Cho; X-ray fluorescence (XRF) imaging, such as x-ray fluorescence computed tomography (XFCT), offers unique capabilities for accurate identification and quantification of metals within the imaging objects. As a result, it has emerged as a promising quantitative imaging modality in recent years, especially in conjunction with metal-based imaging probes. This talk will familiarize the audience with the basic principles of XRF/XFCT imaging. It will also cover the latest development of benchtop XFCT technology. Additionally, the use of metallic nanoparticles such as gold nanoparticles, in conjunction with benchtop XFCT, will be discussed within the context of preclinical multimodal multiplexed molecular imaging. Learning Objectives: To learn the basic principles of XRF/XFCT imaging To learn the latest advances in benchtop XFCT development for preclinical imaging Funding support received from NIH and DOD; Funding support received from GE Healthcare; Funding support received from Siemens AX; Patent royalties received from GE Healthcare; L. Wang, Funding Support: NIH; COI: Microphotoacoustics; S. Cho, Yes: ;NIH/NCI grant R01CA155446 DOD/PCRP grant W81XWH-12-1-0198.« less

Characterizing virus-induced gene silencing at the cellular level with in situ multimodal imaging

DOE PAGES

Burkhow, Sadie J.; Stephens, Nicole M.; Mei, Yu; ...

2018-05-25

Reverse genetic strategies, such as virus-induced gene silencing, are powerful techniques to study gene function. Currently, there are few tools to study the spatial dependence of the consequences of gene silencing at the cellular level. Here, we report the use of multimodal Raman and mass spectrometry imaging to study the cellular-level biochemical changes that occur from silencing the phytoene desaturase ( pds) gene using a Foxtail mosaic virus (FoMV) vector in maize leaves. The multimodal imaging method allows the localized carotenoid distribution to be measured and reveals differences lost in the spatial average when analyzing a carotenoid extraction of themore » whole leaf. The nature of the Raman and mass spectrometry signals are complementary: silencing pds reduces the downstream carotenoid Raman signal and increases the phytoene mass spectrometry signal.« less

Characterizing virus-induced gene silencing at the cellular level with in situ multimodal imaging

DOE Office of Scientific and Technical Information (OSTI.GOV)

Burkhow, Sadie J.; Stephens, Nicole M.; Mei, Yu

Reverse genetic strategies, such as virus-induced gene silencing, are powerful techniques to study gene function. Currently, there are few tools to study the spatial dependence of the consequences of gene silencing at the cellular level. Here, we report the use of multimodal Raman and mass spectrometry imaging to study the cellular-level biochemical changes that occur from silencing the phytoene desaturase ( pds) gene using a Foxtail mosaic virus (FoMV) vector in maize leaves. The multimodal imaging method allows the localized carotenoid distribution to be measured and reveals differences lost in the spatial average when analyzing a carotenoid extraction of themore » whole leaf. The nature of the Raman and mass spectrometry signals are complementary: silencing pds reduces the downstream carotenoid Raman signal and increases the phytoene mass spectrometry signal.« less

Calibration for single multi-mode fiber digital scanning microscopy imaging system

NASA Astrophysics Data System (ADS)

Yin, Zhe; Liu, Guodong; Liu, Bingguo; Gan, Yu; Zhuang, Zhitao; Chen, Fengdong

2015-11-01

Single multimode fiber (MMF) digital scanning imaging system is a development tendency of modern endoscope. We concentrate on the calibration method of the imaging system. Calibration method comprises two processes, forming scanning focused spots and calibrating the couple factors varied with positions. Adaptive parallel coordinate algorithm (APC) is adopted to form the focused spots at the multimode fiber (MMF) output. Compare with other algorithm, APC contains many merits, i.e. rapid speed, small amount calculations and no iterations. The ratio of the optics power captured by MMF to the intensity of the focused spots is called couple factor. We setup the calibration experimental system to form the scanning focused spots and calculate the couple factors for different object positions. The experimental result the couple factor is higher in the center than the edge.

Fast multi-core based multimodal registration of 2D cross-sections and 3D datasets.

PubMed

Scharfe, Michael; Pielot, Rainer; Schreiber, Falk

2010-01-11

Solving bioinformatics tasks often requires extensive computational power. Recent trends in processor architecture combine multiple cores into a single chip to improve overall performance. The Cell Broadband Engine (CBE), a heterogeneous multi-core processor, provides power-efficient and cost-effective high-performance computing. One application area is image analysis and visualisation, in particular registration of 2D cross-sections into 3D image datasets. Such techniques can be used to put different image modalities into spatial correspondence, for example, 2D images of histological cuts into morphological 3D frameworks. We evaluate the CBE-driven PlayStation 3 as a high performance, cost-effective computing platform by adapting a multimodal alignment procedure to several characteristic hardware properties. The optimisations are based on partitioning, vectorisation, branch reducing and loop unrolling techniques with special attention to 32-bit multiplies and limited local storage on the computing units. We show how a typical image analysis and visualisation problem, the multimodal registration of 2D cross-sections and 3D datasets, benefits from the multi-core based implementation of the alignment algorithm. We discuss several CBE-based optimisation methods and compare our results to standard solutions. More information and the source code are available from http://cbe.ipk-gatersleben.de. The results demonstrate that the CBE processor in a PlayStation 3 accelerates computational intensive multimodal registration, which is of great importance in biological/medical image processing. The PlayStation 3 as a low cost CBE-based platform offers an efficient option to conventional hardware to solve computational problems in image processing and bioinformatics.

A Novel Multifunctional Theranostic Liposome Drug Delivery System: Construction, Characterization, and Multimodality MR, Near-infrared Fluorescent and Nuclear Imaging

PubMed Central

Li, Shihong; Goins, Beth; Zhang, Lujun; Bao, Ande

2012-01-01

Liposomes are effective lipid nanoparticle drug delivery systems, which can also be functionalized with non-invasive multimodality imaging agents with each modality providing distinct information and having synergistic advantages in diagnosis, monitoring of disease treatment, and evaluation of liposomal drug pharmacokinetics. We designed and constructed a multifunctional theranostic liposomal drug delivery system, which integrated multimodality magnetic resonance (MR), near-infrared (NIR) fluorescent and nuclear imaging of liposomal drug delivery, and therapy monitoring and prediction. The pre-manufactured liposomes were composed of DSPC/cholesterol/Gd-DOTADSPE/DOTA-DSPE with the molar ratio of 39:35:25:1 and having ammonium sulfate/pH gradient. A lipidized NIR fluorescent tracer, IRDye-DSPE, was effectively post-inserted into the pre-manufactured liposomes. Doxorubicin could be effectively post-loaded into the multifunctional liposomes. The multifunctional doxorubicin-liposomes could also be stably radiolabeled with 99mTc or 64Cu for single photon emission computed tomography (SPECT) or positron emission tomography (PET) imaging, respectively. MR images displayed the high resolution micro-intratumoral distribution of the liposomes in squamous cell carcinoma of head and neck (SCCHN) tumor xenografts in nude rats after intratumoral injection. NIR fluorescent, SPECT and PET images also clearly showed either the high intratumoral retention or distribution of the multifunctional liposomes. This multifunctional drug carrying liposome system is promising for disease theranostics allowing non-invasive multimodality NIR fluorescent, MR, SPECT and PET imaging of their in vivo behavior and capitalizing on the inherent advantages of each modality. PMID:22577859

Analyzing a multimodal biometric system using real and virtual users

NASA Astrophysics Data System (ADS)

Scheidat, Tobias; Vielhauer, Claus

2007-02-01

Three main topics of recent research on multimodal biometric systems are addressed in this article: The lack of sufficiently large multimodal test data sets, the influence of cultural aspects and data protection issues of multimodal biometric data. In this contribution, different possibilities are presented to extend multimodal databases by generating so-called virtual users, which are created by combining single biometric modality data of different users. Comparative tests on databases containing real and virtual users based on a multimodal system using handwriting and speech are presented, to study to which degree the use of virtual multimodal databases allows conclusions with respect to recognition accuracy in comparison to real multimodal data. All tests have been carried out on databases created from donations from three different nationality groups. This allows to review the experimental results both in general and in context of cultural origin. The results show that in most cases the usage of virtual persons leads to lower accuracy than the usage of real users in terms of the measurement applied: the Equal Error Rate. Finally, this article will address the general question how the concept of virtual users may influence the data protection requirements for multimodal evaluation databases in the future.

Multimodal microscopy and the stepwise multi-photon activation fluorescence of melanin

NASA Astrophysics Data System (ADS)

Lai, Zhenhua

The author's work is divided into three aspects: multimodal microscopy, stepwise multi-photon activation fluorescence (SMPAF) of melanin, and customized-profile lenses (CPL) for on-axis laser scanners, which will be introduced respectively. A multimodal microscope provides the ability to image samples with multiple modalities on the same stage, which incorporates the benefits of all modalities. The multimodal microscopes developed in this dissertation are the Keck 3D fusion multimodal microscope 2.0 (3DFM 2.0), upgraded from the old 3DFM with improved performance and flexibility, and the multimodal microscope for targeting small particles (the "Target" system). The control systems developed for both microscopes are low-cost and easy-to-build, with all components off-the-shelf. The control system have not only significantly decreased the complexity and size of the microscope, but also increased the pixel resolution and flexibility. The SMPAF of melanin, activated by a continuous-wave (CW) mode near-infrared (NIR) laser, has potential applications for a low-cost and reliable method of detecting melanin. The photophysics of melanin SMPAF has been studied by theoretical analysis of the excitation process and investigation of the spectra, activation threshold, and photon number absorption of melanin SMPAF. SMPAF images of melanin in mouse hair and skin, mouse melanoma, and human black and white hairs are compared with images taken by conventional multi-photon fluorescence microscopy (MPFM) and confocal reflectance microscopy (CRM). SMPAF images significantly increase specificity and demonstrate the potential to increase sensitivity for melanin detection compared to MPFM images and CRM images. Employing melanin SMPAF imaging to detect melanin inside human skin in vivo has been demonstrated, which proves the effectiveness of melanin detection using SMPAF for medical purposes. Selective melanin ablation with micrometer resolution has been presented using the Target system. Compared to the traditional selective photothermolysis, this method demonstrates higher precision, higher specificity and deeper penetration. Therefore, the SMPAF guided selective ablation of melanin is a promising tool of removing melanin for both medical and cosmetic purposes. Three CPLs have been designed for low-cost linear-motion scanners, low-cost fast spinning scanners and high-precision fast spinning scanners. Each design has been tailored to the industrial manufacturing ability and market demands.

Multimodal brain-tumor segmentation based on Dirichlet process mixture model with anisotropic diffusion and Markov random field prior.

PubMed

Lu, Yisu; Jiang, Jun; Yang, Wei; Feng, Qianjin; Chen, Wufan

2014-01-01

Brain-tumor segmentation is an important clinical requirement for brain-tumor diagnosis and radiotherapy planning. It is well-known that the number of clusters is one of the most important parameters for automatic segmentation. However, it is difficult to define owing to the high diversity in appearance of tumor tissue among different patients and the ambiguous boundaries of lesions. In this study, a nonparametric mixture of Dirichlet process (MDP) model is applied to segment the tumor images, and the MDP segmentation can be performed without the initialization of the number of clusters. Because the classical MDP segmentation cannot be applied for real-time diagnosis, a new nonparametric segmentation algorithm combined with anisotropic diffusion and a Markov random field (MRF) smooth constraint is proposed in this study. Besides the segmentation of single modal brain-tumor images, we developed the algorithm to segment multimodal brain-tumor images by the magnetic resonance (MR) multimodal features and obtain the active tumor and edema in the same time. The proposed algorithm is evaluated using 32 multimodal MR glioma image sequences, and the segmentation results are compared with other approaches. The accuracy and computation time of our algorithm demonstrates very impressive performance and has a great potential for practical real-time clinical use.

Multimodal Brain-Tumor Segmentation Based on Dirichlet Process Mixture Model with Anisotropic Diffusion and Markov Random Field Prior

PubMed Central

Lu, Yisu; Jiang, Jun; Chen, Wufan

2014-01-01

Brain-tumor segmentation is an important clinical requirement for brain-tumor diagnosis and radiotherapy planning. It is well-known that the number of clusters is one of the most important parameters for automatic segmentation. However, it is difficult to define owing to the high diversity in appearance of tumor tissue among different patients and the ambiguous boundaries of lesions. In this study, a nonparametric mixture of Dirichlet process (MDP) model is applied to segment the tumor images, and the MDP segmentation can be performed without the initialization of the number of clusters. Because the classical MDP segmentation cannot be applied for real-time diagnosis, a new nonparametric segmentation algorithm combined with anisotropic diffusion and a Markov random field (MRF) smooth constraint is proposed in this study. Besides the segmentation of single modal brain-tumor images, we developed the algorithm to segment multimodal brain-tumor images by the magnetic resonance (MR) multimodal features and obtain the active tumor and edema in the same time. The proposed algorithm is evaluated using 32 multimodal MR glioma image sequences, and the segmentation results are compared with other approaches. The accuracy and computation time of our algorithm demonstrates very impressive performance and has a great potential for practical real-time clinical use. PMID:25254064

Intraoperative high-field magnetic resonance imaging, multimodal neuronavigation, and intraoperative electrophysiological monitoring-guided surgery for treating supratentorial cavernomas.

PubMed

Li, Fang-Ye; Chen, Xiao-Lei; Xu, Bai-Nan

2016-09-01

To determine the beneficial effects of intraoperative high-field magnetic resonance imaging (MRI), multimodal neuronavigation, and intraoperative electrophysiological monitoring-guided surgery for treating supratentorial cavernomas. Twelve patients with 13 supratentorial cavernomas were prospectively enrolled and operated while using a 1.5 T intraoperative MRI, multimodal neuronavigation, and intraoperative electrophysiological monitoring. All cavernomas were deeply located in subcortical areas or involved critical areas. Intraoperative high-field MRIs were obtained for the intraoperative "visualization" of surrounding eloquent structures, "brain shift" corrections, and navigational plan updates. All cavernomas were successfully resected with guidance from intraoperative MRI, multimodal neuronavigation, and intraoperative electrophysiological monitoring. In 5 cases with supratentorial cavernomas, intraoperative "brain shift" severely deterred locating of the lesions; however, intraoperative MRI facilitated precise locating of these lesions. During long-term (>3 months) follow-up, some or all presenting signs and symptoms improved or resolved in 4 cases, but were unchanged in 7 patients. Intraoperative high-field MRI, multimodal neuronavigation, and intraoperative electrophysiological monitoring are helpful in surgeries for the treatment of small deeply seated subcortical cavernomas.

Performance evaluation of no-reference image quality metrics for face biometric images

NASA Astrophysics Data System (ADS)

Liu, Xinwei; Pedersen, Marius; Charrier, Christophe; Bours, Patrick

2018-03-01

The accuracy of face recognition systems is significantly affected by the quality of face sample images. The recent established standardization proposed several important aspects for the assessment of face sample quality. There are many existing no-reference image quality metrics (IQMs) that are able to assess natural image quality by taking into account similar image-based quality attributes as introduced in the standardization. However, whether such metrics can assess face sample quality is rarely considered. We evaluate the performance of 13 selected no-reference IQMs on face biometrics. The experimental results show that several of them can assess face sample quality according to the system performance. We also analyze the strengths and weaknesses of different IQMs as well as why some of them failed to assess face sample quality. Retraining an original IQM by using face database can improve the performance of such a metric. In addition, the contribution of this paper can be used for the evaluation of IQMs on other biometric modalities; furthermore, it can be used for the development of multimodality biometric IQMs.

Machine learning approaches for integrating clinical and imaging features in late-life depression classification and response prediction.

PubMed

Patel, Meenal J; Andreescu, Carmen; Price, Julie C; Edelman, Kathryn L; Reynolds, Charles F; Aizenstein, Howard J

2015-10-01

Currently, depression diagnosis relies primarily on behavioral symptoms and signs, and treatment is guided by trial and error instead of evaluating associated underlying brain characteristics. Unlike past studies, we attempted to estimate accurate prediction models for late-life depression diagnosis and treatment response using multiple machine learning methods with inputs of multi-modal imaging and non-imaging whole brain and network-based features. Late-life depression patients (medicated post-recruitment) (n = 33) and older non-depressed individuals (n = 35) were recruited. Their demographics and cognitive ability scores were recorded, and brain characteristics were acquired using multi-modal magnetic resonance imaging pretreatment. Linear and nonlinear learning methods were tested for estimating accurate prediction models. A learning method called alternating decision trees estimated the most accurate prediction models for late-life depression diagnosis (87.27% accuracy) and treatment response (89.47% accuracy). The diagnosis model included measures of age, Mini-mental state examination score, and structural imaging (e.g. whole brain atrophy and global white mater hyperintensity burden). The treatment response model included measures of structural and functional connectivity. Combinations of multi-modal imaging and/or non-imaging measures may help better predict late-life depression diagnosis and treatment response. As a preliminary observation, we speculate that the results may also suggest that different underlying brain characteristics defined by multi-modal imaging measures-rather than region-based differences-are associated with depression versus depression recovery because to our knowledge this is the first depression study to accurately predict both using the same approach. These findings may help better understand late-life depression and identify preliminary steps toward personalized late-life depression treatment. Copyright © 2015 John Wiley & Sons, Ltd.

Multimode optical dermoscopy (SkinSpect) analysis for skin with melanocytic nevus

NASA Astrophysics Data System (ADS)

Vasefi, Fartash; MacKinnon, Nicholas; Saager, Rolf; Kelly, Kristen M.; Maly, Tyler; Chave, Robert; Booth, Nicholas; Durkin, Anthony J.; Farkas, Daniel L.

2016-04-01

We have developed a multimode dermoscope (SkinSpect™) capable of illuminating human skin samples in-vivo with spectrally-programmable linearly-polarized light at 33 wavelengths between 468nm and 857 nm. Diffusely reflected photons are separated into collinear and cross-polarized image paths and images captured for each illumination wavelength. In vivo human skin nevi (N = 20) were evaluated with the multimode dermoscope and melanin and hemoglobin concentrations were compared with Spatially Modulated Quantitative Spectroscopy (SMoQS) measurements. Both systems show low correlation between their melanin and hemoglobin concentrations, demonstrating the ability of the SkinSpect™ to separate these molecular signatures and thus act as a biologically plausible device capable of early onset melanoma detection.

Laser Microdissection and Atmospheric Pressure Chemical Ionization Mass Spectrometry Coupled for Multimodal Imaging

DOE Office of Scientific and Technical Information (OSTI.GOV)

Lorenz, Matthias; Ovchinnikova, Olga S; Kertesz, Vilmos

2013-01-01

This paper describes the coupling of ambient laser ablation surface sampling, accomplished using a laser capture microdissection system, with atmospheric pressure chemical ionization mass spectrometry for high spatial resolution multimodal imaging. A commercial laser capture microdissection system was placed in close proximity to a modified ion source of a mass spectrometer designed to allow for sampling of laser ablated material via a transfer tube directly into the ionization region. Rhodamine 6G dye of red sharpie ink in a laser etched pattern as well as cholesterol and phosphatidylcholine in a cerebellum mouse brain thin tissue section were identified and imaged frommore » full scan mass spectra. A minimal spot diameter of 8 m was achieved using the 10X microscope cutting objective with a lateral oversampling pixel resolution of about 3.7 m. Distinguishing between features approximately 13 m apart in a cerebellum mouse brain thin tissue section was demonstrated in a multimodal fashion including co-registered optical and mass spectral chemical images.« less

Design and applications of a multimodality image data warehouse framework.

PubMed

Wong, Stephen T C; Hoo, Kent Soo; Knowlton, Robert C; Laxer, Kenneth D; Cao, Xinhau; Hawkins, Randall A; Dillon, William P; Arenson, Ronald L

2002-01-01

A comprehensive data warehouse framework is needed, which encompasses imaging and non-imaging information in supporting disease management and research. The authors propose such a framework, describe general design principles and system architecture, and illustrate a multimodality neuroimaging data warehouse system implemented for clinical epilepsy research. The data warehouse system is built on top of a picture archiving and communication system (PACS) environment and applies an iterative object-oriented analysis and design (OOAD) approach and recognized data interface and design standards. The implementation is based on a Java CORBA (Common Object Request Broker Architecture) and Web-based architecture that separates the graphical user interface presentation, data warehouse business services, data staging area, and backend source systems into distinct software layers. To illustrate the practicality of the data warehouse system, the authors describe two distinct biomedical applications--namely, clinical diagnostic workup of multimodality neuroimaging cases and research data analysis and decision threshold on seizure foci lateralization. The image data warehouse framework can be modified and generalized for new application domains.

Design and Applications of a Multimodality Image Data Warehouse Framework

PubMed Central

Wong, Stephen T.C.; Hoo, Kent Soo; Knowlton, Robert C.; Laxer, Kenneth D.; Cao, Xinhau; Hawkins, Randall A.; Dillon, William P.; Arenson, Ronald L.

2002-01-01

A comprehensive data warehouse framework is needed, which encompasses imaging and non-imaging information in supporting disease management and research. The authors propose such a framework, describe general design principles and system architecture, and illustrate a multimodality neuroimaging data warehouse system implemented for clinical epilepsy research. The data warehouse system is built on top of a picture archiving and communication system (PACS) environment and applies an iterative object-oriented analysis and design (OOAD) approach and recognized data interface and design standards. The implementation is based on a Java CORBA (Common Object Request Broker Architecture) and Web-based architecture that separates the graphical user interface presentation, data warehouse business services, data staging area, and backend source systems into distinct software layers. To illustrate the practicality of the data warehouse system, the authors describe two distinct biomedical applications—namely, clinical diagnostic workup of multimodality neuroimaging cases and research data analysis and decision threshold on seizure foci lateralization. The image data warehouse framework can be modified and generalized for new application domains. PMID:11971885

A Multimodal Approach to Counselor Supervision.

ERIC Educational Resources Information Center

Ponterotto, Joseph G.; Zander, Toni A.

1984-01-01

Represents an initial effort to apply Lazarus's multimodal approach to a model of counselor supervision. Includes continuously monitoring the trainee's behavior, affect, sensations, images, cognitions, interpersonal functioning, and when appropriate, biological functioning (diet and drugs) in the supervisory process. (LLL)

"I'm Just Trying to Be Tough, Okay": Masculine Performances of Everyday Practices

ERIC Educational Resources Information Center

Collier, Diane R.

2015-01-01

Everyday cultural resources that originate outside school offer possibilities for multimodal creativity and identity play, as children consume, transform and produce multimodal texts. In this study, Kyle, a boy in elementary school, fluidly performed social identities and engaged in playful and parodic rescripting of identities and resources from…

Multimodal Nonlinear Optical Imaging for Sensitive Detection of Multiple Pharmaceutical Solid-State Forms and Surface Transformations.

PubMed

Novakovic, Dunja; Saarinen, Jukka; Rojalin, Tatu; Antikainen, Osmo; Fraser-Miller, Sara J; Laaksonen, Timo; Peltonen, Leena; Isomäki, Antti; Strachan, Clare J

2017-11-07

Two nonlinear imaging modalities, coherent anti-Stokes Raman scattering (CARS) and sum-frequency generation (SFG), were successfully combined for sensitive multimodal imaging of multiple solid-state forms and their changes on drug tablet surfaces. Two imaging approaches were used and compared: (i) hyperspectral CARS combined with principal component analysis (PCA) and SFG imaging and (ii) simultaneous narrowband CARS and SFG imaging. Three different solid-state forms of indomethacin-the crystalline gamma and alpha forms, as well as the amorphous form-were clearly distinguished using both approaches. Simultaneous narrowband CARS and SFG imaging was faster, but hyperspectral CARS and SFG imaging has the potential to be applied to a wider variety of more complex samples. These methodologies were further used to follow crystallization of indomethacin on tablet surfaces under two storage conditions: 30 °C/23% RH and 30 °C/75% RH. Imaging with (sub)micron resolution showed that the approach allowed detection of very early stage surface crystallization. The surfaces progressively crystallized to predominantly (but not exclusively) the gamma form at lower humidity and the alpha form at higher humidity. Overall, this study suggests that multimodal nonlinear imaging is a highly sensitive, solid-state (and chemically) specific, rapid, and versatile imaging technique for understanding and hence controlling (surface) solid-state forms and their complex changes in pharmaceuticals.

A Multi-modal, Discriminative and Spatially Invariant CNN for RGB-D Object Labeling.

PubMed

Asif, Umar; Bennamoun, Mohammed; Sohel, Ferdous

2017-08-30

While deep convolutional neural networks have shown a remarkable success in image classification, the problems of inter-class similarities, intra-class variances, the effective combination of multimodal data, and the spatial variability in images of objects remain to be major challenges. To address these problems, this paper proposes a novel framework to learn a discriminative and spatially invariant classification model for object and indoor scene recognition using multimodal RGB-D imagery. This is achieved through three postulates: 1) spatial invariance - this is achieved by combining a spatial transformer network with a deep convolutional neural network to learn features which are invariant to spatial translations, rotations, and scale changes, 2) high discriminative capability - this is achieved by introducing Fisher encoding within the CNN architecture to learn features which have small inter-class similarities and large intra-class compactness, and 3) multimodal hierarchical fusion - this is achieved through the regularization of semantic segmentation to a multi-modal CNN architecture, where class probabilities are estimated at different hierarchical levels (i.e., imageand pixel-levels), and fused into a Conditional Random Field (CRF)- based inference hypothesis, the optimization of which produces consistent class labels in RGB-D images. Extensive experimental evaluations on RGB-D object and scene datasets, and live video streams (acquired from Kinect) show that our framework produces superior object and scene classification results compared to the state-of-the-art methods.

Application of high-resolution linear Radon transform for Rayleigh-wave dispersive energy imaging and mode separating

USGS Publications Warehouse

Luo, Y.; Xia, J.; Miller, R.D.; Liu, J.; Xu, Y.; Liu, Q.

2008-01-01

Multichannel Analysis of Surface Waves (MASW) analysis is an efficient tool to obtain the vertical shear-wave profile. One of the key steps in the MASW method is to generate an image of dispersive energy in the frequency-velocity domain, so dispersion curves can be determined by picking peaks of dispersion energy. In this paper, we image Rayleigh-wave dispersive energy and separate multimodes from a multichannel record by high-resolution linear Radon transform (LRT). We first introduce Rayleigh-wave dispersive energy imaging by high-resolution LRT. We then show the process of Rayleigh-wave mode separation. Results of synthetic and real-world examples demonstrate that (1) compared with slant stacking algorithm, high-resolution LRT can improve the resolution of images of dispersion energy by more than 50% (2) high-resolution LRT can successfully separate multimode dispersive energy of Rayleigh waves with high resolution; and (3) multimode separation and reconstruction expand frequency ranges of higher mode dispersive energy, which not only increases the investigation depth but also provides a means to accurately determine cut-off frequencies.

PIRATE: pediatric imaging response assessment and targeting environment

NASA Astrophysics Data System (ADS)

Glenn, Russell; Zhang, Yong; Krasin, Matthew; Hua, Chiaho

2010-02-01

By combining the strengths of various imaging modalities, the multimodality imaging approach has potential to improve tumor staging, delineation of tumor boundaries, chemo-radiotherapy regime design, and treatment response assessment in cancer management. To address the urgent needs for efficient tools to analyze large-scale clinical trial data, we have developed an integrated multimodality, functional and anatomical imaging analysis software package for target definition and therapy response assessment in pediatric radiotherapy (RT) patients. Our software provides quantitative tools for automated image segmentation, region-of-interest (ROI) histogram analysis, spatial volume-of-interest (VOI) analysis, and voxel-wise correlation across modalities. To demonstrate the clinical applicability of this software, histogram analyses were performed on baseline and follow-up 18F-fluorodeoxyglucose (18F-FDG) PET images of nine patients with rhabdomyosarcoma enrolled in an institutional clinical trial at St. Jude Children's Research Hospital. In addition, we combined 18F-FDG PET, dynamic-contrast-enhanced (DCE) MR, and anatomical MR data to visualize the heterogeneity in tumor pathophysiology with the ultimate goal of adaptive targeting of regions with high tumor burden. Our software is able to simultaneously analyze multimodality images across multiple time points, which could greatly speed up the analysis of large-scale clinical trial data and validation of potential imaging biomarkers.

Ex vivo catheter-based imaging of coronary atherosclerosis using multimodality OCT and NIRAF excited at 633 nm

PubMed Central

Wang, Hao; Gardecki, Joseph A.; Ughi, Giovanni J.; Jacques, Paulino Vacas; Hamidi, Ehsan; Tearney, Guillermo J.

2015-01-01

While optical coherence tomography (OCT) has been shown to be capable of imaging coronary plaque microstructure, additional chemical/molecular information may be needed in order to determine which lesions are at risk of causing an acute coronary event. In this study, we used a recently developed imaging system and double-clad fiber (DCF) catheter capable of simultaneously acquiring both OCT and red excited near-infrared autofluorescence (NIRAF) images (excitation: 633 nm, emission: 680nm to 900nm). We found that NIRAF is elevated in lesions that contain necrotic core – a feature that is critical for vulnerable plaque diagnosis and that is not readily discriminated by OCT alone. We first utilized a DCF ball lens probe and a bench top setup to acquire en face NIRAF images of aortic plaques ex vivo (n = 20). In addition, we used the OCT-NIRAF system and fully assembled catheters to acquire multimodality images from human coronary arteries (n = 15) prosected from human cadaver hearts (n = 5). Comparison of these images with corresponding histology demonstrated that necrotic core plaques exhibited significantly higher NIRAF intensity than other plaque types. These results suggest that multimodality intracoronary OCT-NIRAF imaging technology may be used in the future to provide improved characterization of coronary artery disease in human patients. PMID:25909020

A vessel segmentation method for multi-modality angiographic images based on multi-scale filtering and statistical models.

PubMed

Lu, Pei; Xia, Jun; Li, Zhicheng; Xiong, Jing; Yang, Jian; Zhou, Shoujun; Wang, Lei; Chen, Mingyang; Wang, Cheng

2016-11-08

Accurate segmentation of blood vessels plays an important role in the computer-aided diagnosis and interventional treatment of vascular diseases. The statistical method is an important component of effective vessel segmentation; however, several limitations discourage the segmentation effect, i.e., dependence of the image modality, uneven contrast media, bias field, and overlapping intensity distribution of the object and background. In addition, the mixture models of the statistical methods are constructed relaying on the characteristics of the image histograms. Thus, it is a challenging issue for the traditional methods to be available in vessel segmentation from multi-modality angiographic images. To overcome these limitations, a flexible segmentation method with a fixed mixture model has been proposed for various angiography modalities. Our method mainly consists of three parts. Firstly, multi-scale filtering algorithm was used on the original images to enhance vessels and suppress noises. As a result, the filtered data achieved a new statistical characteristic. Secondly, a mixture model formed by three probabilistic distributions (two Exponential distributions and one Gaussian distribution) was built to fit the histogram curve of the filtered data, where the expectation maximization (EM) algorithm was used for parameters estimation. Finally, three-dimensional (3D) Markov random field (MRF) were employed to improve the accuracy of pixel-wise classification and posterior probability estimation. To quantitatively evaluate the performance of the proposed method, two phantoms simulating blood vessels with different tubular structures and noises have been devised. Meanwhile, four clinical angiographic data sets from different human organs have been used to qualitatively validate the method. To further test the performance, comparison tests between the proposed method and the traditional ones have been conducted on two different brain magnetic resonance angiography (MRA) data sets. The results of the phantoms were satisfying, e.g., the noise was greatly suppressed, the percentages of the misclassified voxels, i.e., the segmentation error ratios, were no more than 0.3%, and the Dice similarity coefficients (DSCs) were above 94%. According to the opinions of clinical vascular specialists, the vessels in various data sets were extracted with high accuracy since complete vessel trees were extracted while lesser non-vessels and background were falsely classified as vessel. In the comparison experiments, the proposed method showed its superiority in accuracy and robustness for extracting vascular structures from multi-modality angiographic images with complicated background noises. The experimental results demonstrated that our proposed method was available for various angiographic data. The main reason was that the constructed mixture probability model could unitarily classify vessel object from the multi-scale filtered data of various angiography images. The advantages of the proposed method lie in the following aspects: firstly, it can extract the vessels with poor angiography quality, since the multi-scale filtering algorithm can improve the vessel intensity in the circumstance such as uneven contrast media and bias field; secondly, it performed well for extracting the vessels in multi-modality angiographic images despite various signal-noises; and thirdly, it was implemented with better accuracy, and robustness than the traditional methods. Generally, these traits declare that the proposed method would have significant clinical application.

Research of the multimodal brain-tumor segmentation algorithm

NASA Astrophysics Data System (ADS)

Lu, Yisu; Chen, Wufan

2015-12-01

It is well-known that the number of clusters is one of the most important parameters for automatic segmentation. However, it is difficult to define owing to the high diversity in appearance of tumor tissue among different patients and the ambiguous boundaries of lesions. In this study, a nonparametric mixture of Dirichlet process (MDP) model is applied to segment the tumor images, and the MDP segmentation can be performed without the initialization of the number of clusters. A new nonparametric segmentation algorithm combined with anisotropic diffusion and a Markov random field (MRF) smooth constraint is proposed in this study. Besides the segmentation of single modal brain tumor images, we developed the algorithm to segment multimodal brain tumor images by the magnetic resonance (MR) multimodal features and obtain the active tumor and edema in the same time. The proposed algorithm is evaluated and compared with other approaches. The accuracy and computation time of our algorithm demonstrates very impressive performance.

Albumin based versatile multifunctional nanocarriers for cancer therapy: Fabrication, surface modification, multimodal therapeutics and imaging approaches.

PubMed

Kudarha, Ritu R; Sawant, Krutika K

2017-12-01

Albumin is a versatile protein used as a carrier system for cancer therapeutics. As a carrier it can provide tumor specificity, reduce drug related toxicity, maintain therapeutic concentration of the active moiety like drug, gene, peptide, protein etc. for long period of time and also reduce drug related toxicities. Apart from cancer therapy, it is also utilized in the imaging and multimodal therapy of cancer. This review highlights the important properties, structure and types of albumin based nanocarriers with regards to their use for cancer targeting. It also provides brief discussion on methods of preparation of these nanocarriers and their surface modification. Applications of albumin nanocarriers for cancer therapy, gene delivery, imaging, phototherapy and multimodal therapy have also been discussed. This review also provides brief discussion about albumin based marketed nano formulations and those under clinical trials. Copyright © 2017 Elsevier B.V. All rights reserved.

Multimodal MR imaging in hepatic encephalopathy: state of the art.

PubMed

Zhang, Xiao Dong; Zhang, Long Jiang

2018-06-01

Hepatic encephalopathy (HE) is a neurological or neuropsychological complication due to liver failure or portosystemic shunting. The clinical manifestation is highly variable, which can exhibit mild cognitive or motor impairment initially, or gradually progress to a coma, even death, without treatment. Neuroimaging plays a critical role in uncovering the neural mechanism of HE. In particular, multimodality MR imaging is able to assess both structural and functional derangements of the brain with HE in focal or neural network perspectives. In recent years, there has been rapid development in novel MR technologies and applications to investigate the pathophysiological mechanism of HE. Therefore, it is necessary to update the latest MR findings regarding HE by use of multimodality MRI to refine and deepen our understanding of the neural traits in HE. Herein, this review highlights the latest MR imaging findings in HE to refresh our understanding of MRI application in HE.

Spinal metastases: multimodality imaging in diagnosis and stereotactic body radiation therapy planning.

PubMed

Jabehdar Maralani, Pejman; Lo, Simon S; Redmond, Kristin; Soliman, Hany; Myrehaug, Sten; Husain, Zain A; Heyn, Chinthaka; Kapadia, Anish; Chan, Aimee; Sahgal, Arjun

2017-01-01

Due to increased effectiveness of cancer treatments and increasing survival rates, metastatic disease has become more frequent compared to the past, with the spine being the most common site of bony metastases. Diagnostic imaging is an integral part of screening, diagnosis and follow-up of spinal metastases. In this article, we review the principles of multimodality imaging for tumor detection with respect to their value for diagnosis and stereotactic body radiation therapy planning for spinal metastases. We will also review the current international consensus agreement for stereotactic body radiation therapy planning, and the role of imaging in achieving the best possible treatment plan.

Simultaneous in vivo imaging of melanin and lipofuscin in the retina with multimodal photoacoustic ophthalmoscopy

NASA Astrophysics Data System (ADS)

Zhang, Xiangyang; Zhang, Hao F.; Zhou, Lixiang; Jiao, Shuliang

2012-02-01

We combined photoacoustic ophthalmoscopy (PAOM) with autofluorescence imaging for simultaneous in vivo imaging of dual molecular contrasts in the retina using a single light source. The dual molecular contrasts come from melanin and lipofuscin in the retinal pigment epithelium (RPE). Melanin and lipofuscin are two types of pigments and are believed to play opposite roles (protective vs. exacerbate) in the RPE in the aging process. We successfully imaged the retina of pigmented and albino rats at different ages. The experimental results showed that multimodal PAOM system can be a potentially powerful tool in the study of age-related degenerative retinal diseases.

New Finger Biometric Method Using Near Infrared Imaging

PubMed Central

Lee, Eui Chul; Jung, Hyunwoo; Kim, Daeyeoul

2011-01-01

In this paper, we propose a new finger biometric method. Infrared finger images are first captured, and then feature extraction is performed using a modified Gaussian high-pass filter through binarization, local binary pattern (LBP), and local derivative pattern (LDP) methods. Infrared finger images include the multimodal features of finger veins and finger geometries. Instead of extracting each feature using different methods, the modified Gaussian high-pass filter is fully convolved. Therefore, the extracted binary patterns of finger images include the multimodal features of veins and finger geometries. Experimental results show that the proposed method has an error rate of 0.13%. PMID:22163741

Live animal myelin histomorphometry of the spinal cord with video-rate multimodal nonlinear microendoscopy

NASA Astrophysics Data System (ADS)

Bélanger, Erik; Crépeau, Joël; Laffray, Sophie; Vallée, Réal; De Koninck, Yves; Côté, Daniel

2012-02-01

In vivo imaging of cellular dynamics can be dramatically enabling to understand the pathophysiology of nervous system diseases. To fully exploit the power of this approach, the main challenges have been to minimize invasiveness and maximize the number of concurrent optical signals that can be combined to probe the interplay between multiple cellular processes. Label-free coherent anti-Stokes Raman scattering (CARS) microscopy, for example, can be used to follow demyelination in neurodegenerative diseases or after trauma, but myelin imaging alone is not sufficient to understand the complex sequence of events that leads to the appearance of lesions in the white matter. A commercially available microendoscope is used here to achieve minimally invasive, video-rate multimodal nonlinear imaging of cellular processes in live mouse spinal cord. The system allows for simultaneous CARS imaging of myelin sheaths and two-photon excitation fluorescence microendoscopy of microglial cells and axons. Morphometric data extraction at high spatial resolution is also described, with a technique for reducing motion-related imaging artifacts. Despite its small diameter, the microendoscope enables high speed multimodal imaging over wide areas of tissue, yet at resolution sufficient to quantify subtle differences in myelin thickness and microglial motility.

Detail-enhanced multimodality medical image fusion based on gradient minimization smoothing filter and shearing filter.

PubMed

Liu, Xingbin; Mei, Wenbo; Du, Huiqian

2018-02-13

In this paper, a detail-enhanced multimodality medical image fusion algorithm is proposed by using proposed multi-scale joint decomposition framework (MJDF) and shearing filter (SF). The MJDF constructed with gradient minimization smoothing filter (GMSF) and Gaussian low-pass filter (GLF) is used to decompose source images into low-pass layers, edge layers, and detail layers at multiple scales. In order to highlight the detail information in the fused image, the edge layer and the detail layer in each scale are weighted combined into a detail-enhanced layer. As directional filter is effective in capturing salient information, so SF is applied to the detail-enhanced layer to extract geometrical features and obtain directional coefficients. Visual saliency map-based fusion rule is designed for fusing low-pass layers, and the sum of standard deviation is used as activity level measurement for directional coefficients fusion. The final fusion result is obtained by synthesizing the fused low-pass layers and directional coefficients. Experimental results show that the proposed method with shift-invariance, directional selectivity, and detail-enhanced property is efficient in preserving and enhancing detail information of multimodality medical images. Graphical abstract The detailed implementation of the proposed medical image fusion algorithm.

3D multi-scale FCN with random modality voxel dropout learning for Intervertebral Disc Localization and Segmentation from Multi-modality MR Images.

PubMed

Li, Xiaomeng; Dou, Qi; Chen, Hao; Fu, Chi-Wing; Qi, Xiaojuan; Belavý, Daniel L; Armbrecht, Gabriele; Felsenberg, Dieter; Zheng, Guoyan; Heng, Pheng-Ann

2018-04-01

Intervertebral discs (IVDs) are small joints that lie between adjacent vertebrae. The localization and segmentation of IVDs are important for spine disease diagnosis and measurement quantification. However, manual annotation is time-consuming and error-prone with limited reproducibility, particularly for volumetric data. In this work, our goal is to develop an automatic and accurate method based on fully convolutional networks (FCN) for the localization and segmentation of IVDs from multi-modality 3D MR data. Compared with single modality data, multi-modality MR images provide complementary contextual information, which contributes to better recognition performance. However, how to effectively integrate such multi-modality information to generate accurate segmentation results remains to be further explored. In this paper, we present a novel multi-scale and modality dropout learning framework to locate and segment IVDs from four-modality MR images. First, we design a 3D multi-scale context fully convolutional network, which processes the input data in multiple scales of context and then merges the high-level features to enhance the representation capability of the network for handling the scale variation of anatomical structures. Second, to harness the complementary information from different modalities, we present a random modality voxel dropout strategy which alleviates the co-adaption issue and increases the discriminative capability of the network. Our method achieved the 1st place in the MICCAI challenge on automatic localization and segmentation of IVDs from multi-modality MR images, with a mean segmentation Dice coefficient of 91.2% and a mean localization error of 0.62 mm. We further conduct extensive experiments on the extended dataset to validate our method. We demonstrate that the proposed modality dropout strategy with multi-modality images as contextual information improved the segmentation accuracy significantly. Furthermore, experiments conducted on extended data collected from two different time points demonstrate the efficacy of our method on tracking the morphological changes in a longitudinal study. Copyright © 2018 Elsevier B.V. All rights reserved.

Double-Group Particle Swarm Optimization and Its Application in Remote Sensing Image Segmentation

PubMed Central

Shen, Liang; Huang, Xiaotao; Fan, Chongyi

2018-01-01

Particle Swarm Optimization (PSO) is a well-known meta-heuristic. It has been widely used in both research and engineering fields. However, the original PSO generally suffers from premature convergence, especially in multimodal problems. In this paper, we propose a double-group PSO (DG-PSO) algorithm to improve the performance. DG-PSO uses a double-group based evolution framework. The individuals are divided into two groups: an advantaged group and a disadvantaged group. The advantaged group works according to the original PSO, while two new strategies are developed for the disadvantaged group. The proposed algorithm is firstly evaluated by comparing it with the other five popular PSO variants and two state-of-the-art meta-heuristics on various benchmark functions. The results demonstrate that DG-PSO shows a remarkable performance in terms of accuracy and stability. Then, we apply DG-PSO to multilevel thresholding for remote sensing image segmentation. The results show that the proposed algorithm outperforms five other popular algorithms in meta-heuristic-based multilevel thresholding, which verifies the effectiveness of the proposed algorithm. PMID:29724013

Double-Group Particle Swarm Optimization and Its Application in Remote Sensing Image Segmentation.

PubMed

Shen, Liang; Huang, Xiaotao; Fan, Chongyi

2018-05-01

Particle Swarm Optimization (PSO) is a well-known meta-heuristic. It has been widely used in both research and engineering fields. However, the original PSO generally suffers from premature convergence, especially in multimodal problems. In this paper, we propose a double-group PSO (DG-PSO) algorithm to improve the performance. DG-PSO uses a double-group based evolution framework. The individuals are divided into two groups: an advantaged group and a disadvantaged group. The advantaged group works according to the original PSO, while two new strategies are developed for the disadvantaged group. The proposed algorithm is firstly evaluated by comparing it with the other five popular PSO variants and two state-of-the-art meta-heuristics on various benchmark functions. The results demonstrate that DG-PSO shows a remarkable performance in terms of accuracy and stability. Then, we apply DG-PSO to multilevel thresholding for remote sensing image segmentation. The results show that the proposed algorithm outperforms five other popular algorithms in meta-heuristic-based multilevel thresholding, which verifies the effectiveness of the proposed algorithm.

Unifying framework for multimodal brain MRI segmentation based on Hidden Markov Chains.

PubMed

Bricq, S; Collet, Ch; Armspach, J P

2008-12-01

In the frame of 3D medical imaging, accurate segmentation of multimodal brain MR images is of interest for many brain disorders. However, due to several factors such as noise, imaging artifacts, intrinsic tissue variation and partial volume effects, tissue classification remains a challenging task. In this paper, we present a unifying framework for unsupervised segmentation of multimodal brain MR images including partial volume effect, bias field correction, and information given by a probabilistic atlas. Here-proposed method takes into account neighborhood information using a Hidden Markov Chain (HMC) model. Due to the limited resolution of imaging devices, voxels may be composed of a mixture of different tissue types, this partial volume effect is included to achieve an accurate segmentation of brain tissues. Instead of assigning each voxel to a single tissue class (i.e., hard classification), we compute the relative amount of each pure tissue class in each voxel (mixture estimation). Further, a bias field estimation step is added to the proposed algorithm to correct intensity inhomogeneities. Furthermore, atlas priors were incorporated using probabilistic brain atlas containing prior expectations about the spatial localization of different tissue classes. This atlas is considered as a complementary sensor and the proposed method is extended to multimodal brain MRI without any user-tunable parameter (unsupervised algorithm). To validate this new unifying framework, we present experimental results on both synthetic and real brain images, for which the ground truth is available. Comparison with other often used techniques demonstrates the accuracy and the robustness of this new Markovian segmentation scheme.

A low-cost multimodal head-mounted display system for neuroendoscopic surgery.

PubMed

Xu, Xinghua; Zheng, Yi; Yao, Shujing; Sun, Guochen; Xu, Bainan; Chen, Xiaolei

2018-01-01

With rapid advances in technology, wearable devices as head-mounted display (HMD) have been adopted for various uses in medical science, ranging from simply aiding in fitness to assisting surgery. We aimed to investigate the feasibility and practicability of a low-cost multimodal HMD system in neuroendoscopic surgery. A multimodal HMD system, mainly consisted of a HMD with two built-in displays, an action camera, and a laptop computer displaying reconstructed medical images, was developed to assist neuroendoscopic surgery. With this intensively integrated system, the neurosurgeon could freely switch between endoscopic image, three-dimensional (3D) reconstructed virtual endoscopy images, and surrounding environment images. Using a leap motion controller, the neurosurgeon could adjust or rotate the 3D virtual endoscopic images at a distance to better understand the positional relation between lesions and normal tissues at will. A total of 21 consecutive patients with ventricular system diseases underwent neuroendoscopic surgery with the aid of this system. All operations were accomplished successfully, and no system-related complications occurred. The HMD was comfortable to wear and easy to operate. Screen resolution of the HMD was high enough for the neurosurgeon to operate carefully. With the system, the neurosurgeon might get a better comprehension on lesions by freely switching among images of different modalities. The system had a steep learning curve, which meant a quick increment of skill with it. Compared with commercially available surgical assistant instruments, this system was relatively low-cost. The multimodal HMD system is feasible, practical, helpful, and relatively cost efficient in neuroendoscopic surgery.

Fast multi-core based multimodal registration of 2D cross-sections and 3D datasets

PubMed Central

2010-01-01

Background Solving bioinformatics tasks often requires extensive computational power. Recent trends in processor architecture combine multiple cores into a single chip to improve overall performance. The Cell Broadband Engine (CBE), a heterogeneous multi-core processor, provides power-efficient and cost-effective high-performance computing. One application area is image analysis and visualisation, in particular registration of 2D cross-sections into 3D image datasets. Such techniques can be used to put different image modalities into spatial correspondence, for example, 2D images of histological cuts into morphological 3D frameworks. Results We evaluate the CBE-driven PlayStation 3 as a high performance, cost-effective computing platform by adapting a multimodal alignment procedure to several characteristic hardware properties. The optimisations are based on partitioning, vectorisation, branch reducing and loop unrolling techniques with special attention to 32-bit multiplies and limited local storage on the computing units. We show how a typical image analysis and visualisation problem, the multimodal registration of 2D cross-sections and 3D datasets, benefits from the multi-core based implementation of the alignment algorithm. We discuss several CBE-based optimisation methods and compare our results to standard solutions. More information and the source code are available from http://cbe.ipk-gatersleben.de. Conclusions The results demonstrate that the CBE processor in a PlayStation 3 accelerates computational intensive multimodal registration, which is of great importance in biological/medical image processing. The PlayStation 3 as a low cost CBE-based platform offers an efficient option to conventional hardware to solve computational problems in image processing and bioinformatics. PMID:20064262

A Multimodal Imaging Protocol, (123)I/(99)Tc-Sestamibi, SPECT, and SPECT/CT, in Primary Hyperparathyroidism Adds Limited Benefit for Preoperative Localization.

PubMed

Lee, Grace S; McKenzie, Travis J; Mullan, Brian P; Farley, David R; Thompson, Geoffrey B; Richards, Melanie L

2016-03-01

Focused parathyroidectomy in primary hyperparathyroidism (1°HPT) is possible with accurate preoperative localization and intraoperative PTH monitoring (IOPTH). The added benefit of multimodal imaging techniques for operative success is unknown. Patients with 1°HPT, who underwent parathyroidectomy in 2012-2014 at a single institution, were retrospectively reviewed. Only the patients who underwent the standardized multimodal imaging workup consisting of (123)I/(99)Tc-sestamibi subtraction scintigraphy, SPECT, and SPECT/CT were assessed. Of 360 patients who were identified, a curative operation was performed in 96%, using pre-operative imaging and IOPTH. Imaging analysis showed that (123)I/(99)Tc-sestamibi had a sensitivity of 86% (95% CI 82-90%), positive predictive value (PPV) 93%, and accuracy 81%, based on correct lateralization. SPECT had a sensitivity of 77% (95% CI 72-82%), PPV 92% and accuracy 72%. SPECT/CT had a sensitivity of 75% (95% CI 70-80%), PPV of 94%, and accuracy 71%. There were 3 of 45 (7%) patients with negative sestamibi imaging that had an accurate SPECT and SPECT/CT. Of 312 patients (87%) with positive uptake on sestamibi (93% true positive, 7% false positive), concordant findings were present in 86% SPECT and 84% SPECT/CT. In cases where imaging modalities were discordant, but at least one method was true-positive, (123)I/(99)Tc-sestamibi was significantly better than both SPECT and SPECT/CT (p < 0.001). The inclusion of SPECT and SPECT/CT in 1°HPT imaging protocol increases patient cost up to 2.4-fold. (123)I/(99)Tc-sestamibi subtraction imaging is highly sensitive for preoperative localization in 1°HPT. SPECT and SPECT/CT are commonly concordant with (123)I/(99)Tc-sestamibi and rarely increase the sensitivity. Routine inclusion of multimodality imaging technique adds minimal clinical benefit but increases cost to patient in high-volume setting.

Towards Omni-Tomography—Grand Fusion of Multiple Modalities for Simultaneous Interior Tomography

PubMed Central

Wang, Ge; Zhang, Jie; Gao, Hao; Weir, Victor; Yu, Hengyong; Cong, Wenxiang; Xu, Xiaochen; Shen, Haiou; Bennett, James; Furth, Mark; Wang, Yue; Vannier, Michael

2012-01-01

We recently elevated interior tomography from its origin in computed tomography (CT) to a general tomographic principle, and proved its validity for other tomographic modalities including SPECT, MRI, and others. Here we propose “omni-tomography”, a novel concept for the grand fusion of multiple tomographic modalities for simultaneous data acquisition in a region of interest (ROI). Omni-tomography can be instrumental when physiological processes under investigation are multi-dimensional, multi-scale, multi-temporal and multi-parametric. Both preclinical and clinical studies now depend on in vivo tomography, often requiring separate evaluations by different imaging modalities. Over the past decade, two approaches have been used for multimodality fusion: Software based image registration and hybrid scanners such as PET-CT, PET-MRI, and SPECT-CT among others. While there are intrinsic limitations with both approaches, the main obstacle to the seamless fusion of multiple imaging modalities has been the bulkiness of each individual imager and the conflict of their physical (especially spatial) requirements. To address this challenge, omni-tomography is now unveiled as an emerging direction for biomedical imaging and systems biomedicine. PMID:22768108

Rational Design of a Triple Reporter Gene for Multimodality Molecular Imaging

PubMed Central

Hsieh, Ya-Ju; Ke, Chien-Chih; Yeh, Skye Hsin-Hsien; Lin, Chien-Feng; Chen, Fu-Du; Lin, Kang-Ping; Chen, Ran-Chou; Liu, Ren-Shyan

2014-01-01

Multimodality imaging using noncytotoxic triple fusion (TF) reporter genes is an important application for cell-based tracking, drug screening, and therapy. The firefly luciferase (fl), monomeric red fluorescence protein (mrfp), and truncated herpes simplex virus type 1 thymidine kinase SR39 mutant (ttksr39) were fused together to create TF reporter gene constructs with different order. The enzymatic activities of TF protein in vitro and in vivo were determined by luciferase reporter assay, H-FEAU cellular uptake experiment, bioluminescence imaging, and micropositron emission tomography (microPET). The TF construct expressed in H1299 cells possesses luciferase activity and red fluorescence. The tTKSR39 activity is preserved in TF protein and mediates high levels of H-FEAU accumulation and significant cell death from ganciclovir (GCV) prodrug activation. In living animals, the luciferase and tTKSR39 activities of TF protein have also been successfully validated by multimodality imaging systems. The red fluorescence signal is relatively weak for in vivo imaging but may expedite FACS-based selection of TF reporter expressing cells. We have developed an optimized triple fusion reporter construct DsRedm-fl-ttksr39 for more effective and sensitive in vivo animal imaging using fluorescence, bioluminescence, and PET imaging modalities, which may facilitate different fields of biomedical research and applications. PMID:24809057

Advanced Contrast Agents for Multimodal Biomedical Imaging Based on Nanotechnology.

PubMed

Calle, Daniel; Ballesteros, Paloma; Cerdán, Sebastián

2018-01-01

Clinical imaging modalities have reached a prominent role in medical diagnosis and patient management in the last decades. Different image methodologies as Positron Emission Tomography, Single Photon Emission Tomography, X-Rays, or Magnetic Resonance Imaging are in continuous evolution to satisfy the increasing demands of current medical diagnosis. Progress in these methodologies has been favored by the parallel development of increasingly more powerful contrast agents. These are molecules that enhance the intrinsic contrast of the images in the tissues where they accumulate, revealing noninvasively the presence of characteristic molecular targets or differential physiopathological microenvironments. The contrast agent field is currently moving to improve the performance of these molecules by incorporating the advantages that modern nanotechnology offers. These include, mainly, the possibilities to combine imaging and therapeutic capabilities over the same theranostic platform or improve the targeting efficiency in vivo by molecular engineering of the nanostructures. In this review, we provide an introduction to multimodal imaging methods in biomedicine, the sub-nanometric imaging agents previously used and the development of advanced multimodal and theranostic imaging agents based in nanotechnology. We conclude providing some illustrative examples from our own laboratories, including recent progress in theranostic formulations of magnetoliposomes containing ω-3 poly-unsaturated fatty acids to treat inflammatory diseases, or the use of stealth liposomes engineered with a pH-sensitive nanovalve to release their cargo specifically in the acidic extracellular pH microenvironment of tumors.

Superparamagnetic nanoparticles for enhanced magnetic resonance and multimodal imaging

NASA Astrophysics Data System (ADS)

Sikma, Elise Ann Schultz

Magnetic resonance imaging (MRI) is a powerful tool for noninvasive tomographic imaging of biological systems with high spatial and temporal resolution. Superparamagnetic (SPM) nanoparticles have emerged as highly effective MR contrast agents due to their biocompatibility, ease of surface modification and magnetic properties. Conventional nanoparticle contrast agents suffer from difficult synthetic reproducibility, polydisperse sizes and weak magnetism. Numerous synthetic techniques and nanoparticle formulations have been developed to overcome these barriers. However, there are still major limitations in the development of new nanoparticle-based probes for MR and multimodal imaging including low signal amplification and absence of biochemical reporters. To address these issues, a set of multimodal (T2/optical) and dual contrast (T1/T2) nanoparticle probes has been developed. Their unique magnetic properties and imaging capabilities were thoroughly explored. An enzyme-activatable contrast agent is currently being developed as an innovative means for early in vivo detection of cancer at the cellular level. Multimodal probes function by combining the strengths of multiple imaging techniques into a single agent. Co-registration of data obtained by multiple imaging modalities validates the data, enhancing its quality and reliability. A series of T2/optical probes were successfully synthesized by attachment of a fluorescent dye to the surface of different types of nanoparticles. The multimodal nanoparticles generated sufficient MR and fluorescence signal to image transplanted islets in vivo. Dual contrast T1/T2 imaging probes were designed to overcome disadvantages inherent in the individual T1 and T2 components. A class of T1/T2 agents was developed consisting of a gadolinium (III) complex (DTPA chelate or DO3A macrocycle) conjugated to a biocompatible silica-coated metal oxide nanoparticle through a disulfide linker. The disulfide linker has the ability to be reduced in vivo by glutathione, releasing large payloads of signal-enhancing T1 probes into the surrounding environment. Optimization of the agent occurred over three sequential generations, with each generation addressing a new challenge. The result was a T2 nanoparticle containing high levels of conjugated T1 complex demonstrating enhanced MR relaxation properties. The probes created here have the potential to play a key role in the advancement of nanoparticle-based agents in biomedical MRI applications.

Multimodality imaging of hepato-biliary disorders in pregnancy: a pictorial essay.

PubMed

Ong, Eugene M W; Drukteinis, Jennifer S; Peters, Hope E; Mortelé, Koenraad J

2009-09-01

Hepato-biliary disorders are rare complications of pregnancy, but they may be severe, with high fetal and maternal morbidity and mortality. Imaging is, therefore, essential in the rapid diagnosis of some of these conditions so that appropriate, life-saving treatment can be administered. This pictorial essay illustrates the multimodality imaging features of pregnancy-induced hepato-biliary disorders, such as acute fatty liver of pregnancy, preeclamsia and eclampsia, and HELLP syndrome, as well as those conditions which occur in pregnancy but are not unique to it, such as viral hepatitis, Budd-Chiari syndrome, focal hepatic lesions, biliary sludge, cholecystolithiasis, and choledocholithiasis.

Automated diagnosis of prostate cancer in multi-parametric MRI based on multimodal convolutional neural networks

NASA Astrophysics Data System (ADS)

Le, Minh Hung; Chen, Jingyu; Wang, Liang; Wang, Zhiwei; Liu, Wenyu; (Tim Cheng, Kwang-Ting; Yang, Xin

2017-08-01

Automated methods for prostate cancer (PCa) diagnosis in multi-parametric magnetic resonance imaging (MP-MRIs) are critical for alleviating requirements for interpretation of radiographs while helping to improve diagnostic accuracy (Artan et al 2010 IEEE Trans. Image Process. 19 2444-55, Litjens et al 2014 IEEE Trans. Med. Imaging 33 1083-92, Liu et al 2013 SPIE Medical Imaging (International Society for Optics and Photonics) p 86701G, Moradi et al 2012 J. Magn. Reson. Imaging 35 1403-13, Niaf et al 2014 IEEE Trans. Image Process. 23 979-91, Niaf et al 2012 Phys. Med. Biol. 57 3833, Peng et al 2013a SPIE Medical Imaging (International Society for Optics and Photonics) p 86701H, Peng et al 2013b Radiology 267 787-96, Wang et al 2014 BioMed. Res. Int. 2014). This paper presents an automated method based on multimodal convolutional neural networks (CNNs) for two PCa diagnostic tasks: (1) distinguishing between cancerous and noncancerous tissues and (2) distinguishing between clinically significant (CS) and indolent PCa. Specifically, our multimodal CNNs effectively fuse apparent diffusion coefficients (ADCs) and T2-weighted MP-MRI images (T2WIs). To effectively fuse ADCs and T2WIs we design a new similarity loss function to enforce consistent features being extracted from both ADCs and T2WIs. The similarity loss is combined with the conventional classification loss functions and integrated into the back-propagation procedure of CNN training. The similarity loss enables better fusion results than existing methods as the feature learning processes of both modalities are mutually guided, jointly facilitating CNN to ‘see’ the true visual patterns of PCa. The classification results of multimodal CNNs are further combined with the results based on handcrafted features using a support vector machine classifier. To achieve a satisfactory accuracy for clinical use, we comprehensively investigate three critical factors which could greatly affect the performance of our multimodal CNNs but have not been carefully studied previously. (1) Given limited training data, how can these be augmented in sufficient numbers and variety for fine-tuning deep CNN networks for PCa diagnosis? (2) How can multimodal MP-MRI information be effectively combined in CNNs? (3) What is the impact of different CNN architectures on the accuracy of PCa diagnosis? Experimental results on extensive clinical data from 364 patients with a total of 463 PCa lesions and 450 identified noncancerous image patches demonstrate that our system can achieve a sensitivity of 89.85% and a specificity of 95.83% for distinguishing cancer from noncancerous tissues and a sensitivity of 100% and a specificity of 76.92% for distinguishing indolent PCa from CS PCa. This result is significantly superior to the state-of-the-art method relying on handcrafted features.

The evolution of gadolinium based contrast agents: from single-modality to multi-modality

NASA Astrophysics Data System (ADS)

Zhang, Li; Liu, Ruiqing; Peng, Hui; Li, Penghui; Xu, Zushun; Whittaker, Andrew K.

2016-05-01

Gadolinium-based contrast agents are extensively used as magnetic resonance imaging (MRI) contrast agents due to their outstanding signal enhancement and ease of chemical modification. However, it is increasingly recognized that information obtained from single modal molecular imaging cannot satisfy the higher requirements on the efficiency and accuracy for clinical diagnosis and medical research, due to its limitation and default rooted in single molecular imaging technique itself. To compensate for the deficiencies of single function magnetic resonance imaging contrast agents, the combination of multi-modality imaging has turned to be the research hotpot in recent years. This review presents an overview on the recent developments of the functionalization of gadolinium-based contrast agents, and their application in biomedicine applications.

A Pretargeted Approach for the Multimodal PET/NIRF Imaging of Colorectal Cancer.

PubMed

Adumeau, Pierre; Carnazza, Kathryn E; Brand, Christian; Carlin, Sean D; Reiner, Thomas; Agnew, Brian J; Lewis, Jason S; Zeglis, Brian M

2016-01-01

The complementary nature of positron emission tomography (PET) and near-infrared fluorescence (NIRF) imaging makes the development of strategies for the multimodal PET/NIRF imaging of cancer a very enticing prospect. Indeed, in the context of colorectal cancer, a single multimodal PET/NIRF imaging agent could be used to stage the disease, identify candidates for surgical intervention, and facilitate the image-guided resection of the disease. While antibodies have proven to be highly effective vectors for the delivery of radioisotopes and fluorophores to malignant tissues, the use of radioimmunoconjugates labeled with long-lived nuclides such as 89 Zr poses two important clinical complications: high radiation doses to the patient and the need for significant lag time between imaging and surgery. In vivo pretargeting strategies that decouple the targeting vector from the radioactivity at the time of injection have the potential to circumvent these issues by facilitating the use of positron-emitting radioisotopes with far shorter half-lives. Here, we report the synthesis, characterization, and in vivo validation of a pretargeted strategy for the multimodal PET and NIRF imaging of colorectal carcinoma. This approach is based on the rapid and bioorthogonal ligation between a trans -cyclooctene- and fluorophore-bearing immunoconjugate of the huA33 antibody (huA33-Dye800-TCO) and a 64 Cu-labeled tetrazine radioligand ( 64 Cu-Tz-SarAr). In vivo imaging experiments in mice bearing A33 antigen-expressing SW1222 colorectal cancer xenografts clearly demonstrate that this approach enables the non-invasive visualization of tumors and the image-guided resection of malignant tissue, all at only a fraction of the radiation dose created by a directly labeled radioimmunoconjugate. Additional in vivo experiments in peritoneal and patient-derived xenograft models of colorectal carcinoma reinforce the efficacy of this methodology and underscore its potential as an innovative and useful clinical tool.

Acute imaging does not improve ASTRAL score's accuracy despite having a prognostic value.

PubMed

Ntaios, George; Papavasileiou, Vasileios; Faouzi, Mohamed; Vanacker, Peter; Wintermark, Max; Michel, Patrik

2014-10-01

The ASTRAL score was recently shown to reliably predict three-month functional outcome in patients with acute ischemic stroke. The study aims to investigate whether information from multimodal imaging increases ASTRAL score's accuracy. All patients registered in the ASTRAL registry until March 2011 were included. In multivariate logistic-regression analyses, we added covariates derived from parenchymal, vascular, and perfusion imaging to the 6-parameter model of the ASTRAL score. If a specific imaging covariate remained an independent predictor of three-month modified Rankin score>2, the area-under-the-curve (AUC) of this new model was calculated and compared with ASTRAL score's AUC. We also performed similar logistic regression analyses in arbitrarily chosen patient subgroups. When added to the ASTRAL score, the following covariates on admission computed tomography/magnetic resonance imaging-based multimodal imaging were not significant predictors of outcome: any stroke-related acute lesion, any nonstroke-related lesions, chronic/subacute stroke, leukoaraiosis, significant arterial pathology in ischemic territory on computed tomography angiography/magnetic resonance angiography/Doppler, significant intracranial arterial pathology in ischemic territory, and focal hypoperfusion on perfusion-computed tomography. The Alberta Stroke Program Early CT score on plain imaging and any significant extracranial arterial pathology on computed tomography angiography/magnetic resonance angiography/Doppler were independent predictors of outcome (odds ratio: 0·93, 95% CI: 0·87-0·99 and odds ratio: 1·49, 95% CI: 1·08-2·05, respectively) but did not increase ASTRAL score's AUC (0·849 vs. 0·850, and 0·8563 vs. 0·8564, respectively). In exploratory analyses in subgroups of different prognosis, age or stroke severity, no covariate was found to increase ASTRAL score's AUC, either. The addition of information derived from multimodal imaging does not increase ASTRAL score's accuracy to predict functional outcome despite having an independent prognostic value. More selected radiological parameters applied in specific subgroups of stroke patients may add prognostic value of multimodal imaging. © 2014 World Stroke Organization.

Synthesis and radiolabeling of a somatostatin analog for multimodal imaging

NASA Astrophysics Data System (ADS)

Edwards, W. Barry; Liang, Kexian; Xu, Baogang; Anderson, Carolyn J.; Achilefu, Samuel

2006-02-01

A new multimodal imaging agent for imaging the somatostatin receptor has been synthesized and evaluated in vitro and in vivo. A somatostatin analog, conjugated to both 1,4,7,10-tetraazacyclododecane-1,4,7,10-tetraaceticacid (DOTA) and cypate (BS-296), was synthesized entirely on the solid phase (Fmoc) and purified by RP-HPLC. DOTA was added as a ligand for radiometals such as 64Cu or 177Lu for either radio-imaging or radiotherapy respectively. Cytate, a cypatesomatostatin analog conjugate, has previously demonstrated the ability to visualize somatostatin receptor rich tumor xenografts and natural organs by optical imaging techniques. BS-296 exhibited low nanomolar inhibitory capacity toward the binding of radiolabeled somatostatin analogs in cell membranes enriched in the somatostatin receptor, demonstrating the high affinity of this multimodal imaging peptide and indicating its potential as a molecular imaging agent. 64Cu, an isotope for diagnostic imaging and radiotherapy, was selected as the isotope for radiolabeling BS-296. BS-296 was radiolabeled with 64Cu in high specific activity (200 μCi/μg) in 90% radiochemical yield. Addition of 2,5-dihydroxybenzoic acid (gentisic acid) prevented radiolysis of the sample, allowing for study of the 64Cu -BS-296 the day following radiolabeling. Furthermore, inclusion of DMSO at a level of 20% was found not to interfere with radiolabeling yields and prevented the adherence of 64Cu -BS-296 to the walls of the reaction vessel.

A multimodal imaging approach enables in vivo assessment of antifungal treatment in a mouse model of invasive pulmonary aspergillosis.

PubMed

Poelmans, Jennifer; Himmelreich, Uwe; Vanherp, Liesbeth; Zhai, Luca; Hillen, Amy; Holvoet, Bryan; Belderbos, Sarah; Brock, Matthias; Maertens, Johan; Velde, Greetje Vande; Lagrou, Katrien

2018-05-14

Aspergillus fumigatus causes life-threatening lung infections in immunocompromised patients. Mouse models are extensively used in research to assess the in vivo efficacy of antifungals. In recent years, there has been an increasing interest in the use of non-invasive imaging techniques to evaluate experimental infections. However, single imaging modalities have limitations concerning the type of information they can provide. In this study, magnetic resonance imaging and bioluminescence imaging were combined to obtain longitudinal information on the extent of developing lesions and fungal load in a leucopenic mouse model of IPA. This multimodal imaging approach was used to assess changes occurring within lungs of infected mice receiving voriconazole treatment starting at different time points after infection. Results showed that IPA development depends on the inoculum size used to infect animals and that disease can be successfully prevented or treated by initiating intervention during early stages of infection. Furthermore, we demonstrated that reduction of the fungal load is not necessarily associated with the disappearance of lesions on anatomical lung images, especially when antifungal treatment coincides with immune recovery. In conclusion, multimodal imaging allows to investigate different aspects of disease progression or recovery by providing complementary information on dynamic processes, which are highly useful for assessing the efficacy of (novel) therapeutic compounds in a time- and labor-efficient manner. Copyright © 2018 American Society for Microbiology.

Enhancing image classification models with multi-modal biomarkers

NASA Astrophysics Data System (ADS)

Caban, Jesus J.; Liao, David; Yao, Jianhua; Mollura, Daniel J.; Gochuico, Bernadette; Yoo, Terry

2011-03-01

Currently, most computer-aided diagnosis (CAD) systems rely on image analysis and statistical models to diagnose, quantify, and monitor the progression of a particular disease. In general, CAD systems have proven to be effective at providing quantitative measurements and assisting physicians during the decision-making process. As the need for more flexible and effective CADs continues to grow, questions about how to enhance their accuracy have surged. In this paper, we show how statistical image models can be augmented with multi-modal physiological values to create more robust, stable, and accurate CAD systems. In particular, this paper demonstrates how highly correlated blood and EKG features can be treated as biomarkers and used to enhance image classification models designed to automatically score subjects with pulmonary fibrosis. In our results, a 3-5% improvement was observed when comparing the accuracy of CADs that use multi-modal biomarkers with those that only used image features. Our results show that lab values such as Erythrocyte Sedimentation Rate and Fibrinogen, as well as EKG measurements such as QRS and I:40, are statistically significant and can provide valuable insights about the severity of the pulmonary fibrosis disease.

Large Margin Multi-Modal Multi-Task Feature Extraction for Image Classification.

PubMed

Yong Luo; Yonggang Wen; Dacheng Tao; Jie Gui; Chao Xu

2016-01-01

The features used in many image analysis-based applications are frequently of very high dimension. Feature extraction offers several advantages in high-dimensional cases, and many recent studies have used multi-task feature extraction approaches, which often outperform single-task feature extraction approaches. However, most of these methods are limited in that they only consider data represented by a single type of feature, even though features usually represent images from multiple modalities. We, therefore, propose a novel large margin multi-modal multi-task feature extraction (LM3FE) framework for handling multi-modal features for image classification. In particular, LM3FE simultaneously learns the feature extraction matrix for each modality and the modality combination coefficients. In this way, LM3FE not only handles correlated and noisy features, but also utilizes the complementarity of different modalities to further help reduce feature redundancy in each modality. The large margin principle employed also helps to extract strongly predictive features, so that they are more suitable for prediction (e.g., classification). An alternating algorithm is developed for problem optimization, and each subproblem can be efficiently solved. Experiments on two challenging real-world image data sets demonstrate the effectiveness and superiority of the proposed method.

Multimodal hyperspectral optical microscopy

DOE PAGES

Novikova, Irina V.; Smallwood, Chuck R.; Gong, Yu; ...

2017-09-02

We describe a unique and convenient approach to multimodal hyperspectral optical microscopy, herein achieved by coupling a portable and transferable hyperspectral imager to various optical microscopes. The experimental and data analysis schemes involved in recording spectrally and spatially resolved fluorescence, dark field, and optical absorption micrographs are illustrated through prototypical measurements targeting selected model systems. Namely, hyperspectral fluorescence micrographs of isolated fluorescent beads are employed to ensure spectral calibration of our detector and to gauge the attainable spatial resolution of our measurements; the recorded images are diffraction-limited. Moreover, spatially over-sampled absorption spectroscopy of a single lipid (18:1 Liss Rhod PE)more » layer reveals that optical densities on the order of 10-3 may be resolved by spatially averaging the recorded optical signatures. We also briefly illustrate two applications of our setup in the general areas of plasmonics and cell biology. Most notably, we deploy hyperspectral optical absorption microscopy to identify and image algal pigments within a single live Tisochrysis lutea cell. Overall, this work paves the way for multimodal multidimensional spectral imaging measurements spanning the realms of several scientific disciples.« less

Colocalization of cellular nanostructure using confocal fluorescence and partial wave spectroscopy.

PubMed

Chandler, John E; Stypula-Cyrus, Yolanda; Almassalha, Luay; Bauer, Greta; Bowen, Leah; Subramanian, Hariharan; Szleifer, Igal; Backman, Vadim

2017-03-01

A new multimodal confocal microscope has been developed, which includes a parallel Partial Wave Spectroscopic (PWS) microscopy path. This combination of modalities allows molecular-specific sensing of nanoscale intracellular structure using fluorescent labels. Combining molecular specificity and sensitivity to nanoscale structure allows localization of nanostructural intracellular changes, which is critical for understanding the mechanisms of diseases such as cancer. To demonstrate the capabilities of this multimodal instrument, we imaged HeLa cells treated with valinomycin, a potassium ionophore that uncouples oxidative phosphorylation. Colocalization of fluorescence images of the nuclei (Hoechst 33342) and mitochondria (anti-mitochondria conjugated to Alexa Fluor 488) with PWS measurements allowed us to detect a significant decrease in nuclear nanoscale heterogeneity (Σ), while no significant change in Σ was observed at mitochondrial sites. In addition, application of the new multimodal imaging approach was demonstrated on human buccal samples prepared using a cancer screening protocol. These images demonstrate that nanoscale intracellular structure can be studied in healthy and diseased cells at molecular-specific sites. © 2017 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim.

Simultaneous fluorescence and quantitative phase microscopy with single-pixel detectors

NASA Astrophysics Data System (ADS)

Liu, Yang; Suo, Jinli; Zhang, Yuanlong; Dai, Qionghai

2018-02-01

Multimodal microscopy offers high flexibilities for biomedical observation and diagnosis. Conventional multimodal approaches either use multiple cameras or a single camera spatially multiplexing different modes. The former needs expertise demanding alignment and the latter suffers from limited spatial resolution. Here, we report an alignment-free full-resolution simultaneous fluorescence and quantitative phase imaging approach using single-pixel detectors. By combining reference-free interferometry with single-pixel detection, we encode the phase and fluorescence of the sample in two detection arms at the same time. Then we employ structured illumination and the correlated measurements between the sample and the illuminations for reconstruction. The recovered fluorescence and phase images are inherently aligned thanks to single-pixel detection. To validate the proposed method, we built a proof-of-concept setup for first imaging the phase of etched glass with the depth of a few hundred nanometers and then imaging the fluorescence and phase of the quantum dot drop. This method holds great potential for multispectral fluorescence microscopy with additional single-pixel detectors or a spectrometer. Besides, this cost-efficient multimodal system might find broad applications in biomedical science and neuroscience.

Multimodal hyperspectral optical microscopy

DOE Office of Scientific and Technical Information (OSTI.GOV)

Novikova, Irina V.; Smallwood, Chuck R.; Gong, Yu

We describe a unique and convenient approach to multimodal hyperspectral optical microscopy, herein achieved by coupling a portable and transferable hyperspectral imager to various optical microscopes. The experimental and data analysis schemes involved in recording spectrally and spatially resolved fluorescence, dark field, and optical absorption micrographs are illustrated through prototypical measurements targeting selected model systems. Namely, hyperspectral fluorescence micrographs of isolated fluorescent beads are employed to ensure spectral calibration of our detector and to gauge the attainable spatial resolution of our measurements; the recorded images are diffraction-limited. Moreover, spatially over-sampled absorption spectroscopy of a single lipid (18:1 Liss Rhod PE)more » layer reveals that optical densities on the order of 10-3 may be resolved by spatially averaging the recorded optical signatures. We also briefly illustrate two applications of our setup in the general areas of plasmonics and cell biology. Most notably, we deploy hyperspectral optical absorption microscopy to identify and image algal pigments within a single live Tisochrysis lutea cell. Overall, this work paves the way for multimodal multidimensional spectral imaging measurements spanning the realms of several scientific disciples.« less

Love that Book: Multimodal Response to Literature

ERIC Educational Resources Information Center

Dalton, Bridget; Grisham, Dana L.

2013-01-01

Composing with different modes--image, sound, video and the written word--to respond to and analyze literary and informational text helps students develop as readers and digital communicators. This article showcases five multimodal strategies for engaging children in rich literature-based learning using digital tools and Internet resources.

Depth-resolved imaging of colon tumor using optical coherence tomography and fluorescence laminar optical tomography (Conference Presentation)

NASA Astrophysics Data System (ADS)

Tang, Qinggong; Frank, Aaron; Wang, Jianting; Chen, Chao-wei; Jin, Lily; Lin, Jon; Chan, Joanne M.; Chen, Yu

2016-03-01

Early detection of neoplastic changes remains a critical challenge in clinical cancer diagnosis and treatment. Many cancers arise from epithelial layers such as those of the gastrointestinal (GI) tract. Current standard endoscopic technology is unable to detect those subsurface lesions. Since cancer development is associated with both morphological and molecular alterations, imaging technologies that can quantitative image tissue's morphological and molecular biomarkers and assess the depth extent of a lesion in real time, without the need for tissue excision, would be a major advance in GI cancer diagnostics and therapy. In this research, we investigated the feasibility of multi-modal optical imaging including high-resolution optical coherence tomography (OCT) and depth-resolved high-sensitivity fluorescence laminar optical tomography (FLOT) for structural and molecular imaging. APC (adenomatous polyposis coli) mice model were imaged using OCT and FLOT and the correlated histopathological diagnosis was obtained. Quantitative structural (the scattering coefficient) and molecular imaging parameters (fluorescence intensity) from OCT and FLOT images were developed for multi-parametric analysis. This multi-modal imaging method has demonstrated the feasibility for more accurate diagnosis with 87.4% (87.3%) for sensitivity (specificity) which gives the most optimal diagnosis (the largest area under receiver operating characteristic (ROC) curve). This project results in a new non-invasive multi-modal imaging platform for improved GI cancer detection, which is expected to have a major impact on detection, diagnosis, and characterization of GI cancers, as well as a wide range of epithelial cancers.

WE-H-206-03: Promises and Challenges of Benchtop X-Ray Fluorescence CT (XFCT) for Quantitative in Vivo Imaging

DOE Office of Scientific and Technical Information (OSTI.GOV)

Cho, S.

Lihong V. Wang: Photoacoustic tomography (PAT), combining non-ionizing optical and ultrasonic waves via the photoacoustic effect, provides in vivo multiscale functional, metabolic, and molecular imaging. Broad applications include imaging of the breast, brain, skin, esophagus, colon, vascular system, and lymphatic system in humans or animals. Light offers rich contrast but does not penetrate biological tissue in straight paths as x-rays do. Consequently, high-resolution pure optical imaging (e.g., confocal microscopy, two-photon microscopy, and optical coherence tomography) is limited to penetration within the optical diffusion limit (∼1 mm in the skin). Ultrasonic imaging, on the contrary, provides fine spatial resolution but suffersmore » from both poor contrast in early-stage tumors and strong speckle artifacts. In PAT, pulsed laser light penetrates tissue and generates a small but rapid temperature rise, which induces emission of ultrasonic waves due to thermoelastic expansion. The ultrasonic waves, orders of magnitude less scattering than optical waves, are then detected to form high-resolution images of optical absorption at depths up to 7 cm, conquering the optical diffusion limit. PAT is the only modality capable of imaging across the length scales of organelles, cells, tissues, and organs (up to whole-body small animals) with consistent contrast. This rapidly growing technology promises to enable multiscale biological research and accelerate translation from microscopic laboratory discoveries to macroscopic clinical practice. PAT may also hold the key to label-free early detection of cancer by in vivo quantification of hypermetabolism, the quintessential hallmark of malignancy. Learning Objectives: To understand the contrast mechanism of PAT To understand the multiscale applications of PAT Benjamin M. W. Tsui: Multi-modality molecular imaging instrumentation and techniques have been major developments in small animal imaging that has contributed significantly to biomedical research during the past decade. The initial development was an extension of clinical PET/CT and SPECT/CT from human to small animals and combine the unique functional information obtained from PET and SPECT with anatomical information provided by the CT in registered multi-modality images. The requirements to image a mouse whose size is an order of magnitude smaller than that of a human have spurred advances in new radiation detector technologies, novel imaging system designs and special image reconstruction and processing techniques. Examples are new detector materials and designs with high intrinsic resolution, multi-pinhole (MPH) collimator design for much improved resolution and detection efficiency compared to the conventional collimator designs in SPECT, 3D high-resolution and artifact-free MPH and sparse-view image reconstruction techniques, and iterative image reconstruction methods with system response modeling for resolution recovery and image noise reduction for much improved image quality. The spatial resolution of PET and SPECT has improved from ∼6–12 mm to ∼1 mm a few years ago to sub-millimeter today. A recent commercial small animal SPECT system has achieved a resolution of ∼0.25 mm which surpasses that of a state-of-art PET system whose resolution is limited by the positron range. More recently, multimodality SA PET/MRI and SPECT/MRI systems have been developed in research laboratories. Also, multi-modality SA imaging systems that include other imaging modalities such as optical and ultrasound are being actively pursued. In this presentation, we will provide a review of the development, recent advances and future outlook of multi-modality molecular imaging of small animals. Learning Objectives: To learn about the two major multi-modality molecular imaging techniques of small animals. To learn about the spatial resolution achievable by the molecular imaging systems for small animal today. To learn about the new multi-modality imaging instrumentation and techniques that are being developed. Sang Hyun Cho; X-ray fluorescence (XRF) imaging, such as x-ray fluorescence computed tomography (XFCT), offers unique capabilities for accurate identification and quantification of metals within the imaging objects. As a result, it has emerged as a promising quantitative imaging modality in recent years, especially in conjunction with metal-based imaging probes. This talk will familiarize the audience with the basic principles of XRF/XFCT imaging. It will also cover the latest development of benchtop XFCT technology. Additionally, the use of metallic nanoparticles such as gold nanoparticles, in conjunction with benchtop XFCT, will be discussed within the context of preclinical multimodal multiplexed molecular imaging. Learning Objectives: To learn the basic principles of XRF/XFCT imaging To learn the latest advances in benchtop XFCT development for preclinical imaging Funding support received from NIH and DOD; Funding support received from GE Healthcare; Funding support received from Siemens AX; Patent royalties received from GE Healthcare; L. Wang, Funding Support: NIH; COI: Microphotoacoustics; S. Cho, Yes: ;NIH/NCI grant R01CA155446 DOD/PCRP grant W81XWH-12-1-0198.« less

WE-H-206-01: Photoacoustic Tomography: Multiscale Imaging From Organelles to Patients by Ultrasonically Beating the Optical Diffusion Limit

DOE Office of Scientific and Technical Information (OSTI.GOV)

Wang, L.

Lihong V. Wang: Photoacoustic tomography (PAT), combining non-ionizing optical and ultrasonic waves via the photoacoustic effect, provides in vivo multiscale functional, metabolic, and molecular imaging. Broad applications include imaging of the breast, brain, skin, esophagus, colon, vascular system, and lymphatic system in humans or animals. Light offers rich contrast but does not penetrate biological tissue in straight paths as x-rays do. Consequently, high-resolution pure optical imaging (e.g., confocal microscopy, two-photon microscopy, and optical coherence tomography) is limited to penetration within the optical diffusion limit (∼1 mm in the skin). Ultrasonic imaging, on the contrary, provides fine spatial resolution but suffersmore » from both poor contrast in early-stage tumors and strong speckle artifacts. In PAT, pulsed laser light penetrates tissue and generates a small but rapid temperature rise, which induces emission of ultrasonic waves due to thermoelastic expansion. The ultrasonic waves, orders of magnitude less scattering than optical waves, are then detected to form high-resolution images of optical absorption at depths up to 7 cm, conquering the optical diffusion limit. PAT is the only modality capable of imaging across the length scales of organelles, cells, tissues, and organs (up to whole-body small animals) with consistent contrast. This rapidly growing technology promises to enable multiscale biological research and accelerate translation from microscopic laboratory discoveries to macroscopic clinical practice. PAT may also hold the key to label-free early detection of cancer by in vivo quantification of hypermetabolism, the quintessential hallmark of malignancy. Learning Objectives: To understand the contrast mechanism of PAT To understand the multiscale applications of PAT Benjamin M. W. Tsui: Multi-modality molecular imaging instrumentation and techniques have been major developments in small animal imaging that has contributed significantly to biomedical research during the past decade. The initial development was an extension of clinical PET/CT and SPECT/CT from human to small animals and combine the unique functional information obtained from PET and SPECT with anatomical information provided by the CT in registered multi-modality images. The requirements to image a mouse whose size is an order of magnitude smaller than that of a human have spurred advances in new radiation detector technologies, novel imaging system designs and special image reconstruction and processing techniques. Examples are new detector materials and designs with high intrinsic resolution, multi-pinhole (MPH) collimator design for much improved resolution and detection efficiency compared to the conventional collimator designs in SPECT, 3D high-resolution and artifact-free MPH and sparse-view image reconstruction techniques, and iterative image reconstruction methods with system response modeling for resolution recovery and image noise reduction for much improved image quality. The spatial resolution of PET and SPECT has improved from ∼6–12 mm to ∼1 mm a few years ago to sub-millimeter today. A recent commercial small animal SPECT system has achieved a resolution of ∼0.25 mm which surpasses that of a state-of-art PET system whose resolution is limited by the positron range. More recently, multimodality SA PET/MRI and SPECT/MRI systems have been developed in research laboratories. Also, multi-modality SA imaging systems that include other imaging modalities such as optical and ultrasound are being actively pursued. In this presentation, we will provide a review of the development, recent advances and future outlook of multi-modality molecular imaging of small animals. Learning Objectives: To learn about the two major multi-modality molecular imaging techniques of small animals. To learn about the spatial resolution achievable by the molecular imaging systems for small animal today. To learn about the new multi-modality imaging instrumentation and techniques that are being developed. Sang Hyun Cho; X-ray fluorescence (XRF) imaging, such as x-ray fluorescence computed tomography (XFCT), offers unique capabilities for accurate identification and quantification of metals within the imaging objects. As a result, it has emerged as a promising quantitative imaging modality in recent years, especially in conjunction with metal-based imaging probes. This talk will familiarize the audience with the basic principles of XRF/XFCT imaging. It will also cover the latest development of benchtop XFCT technology. Additionally, the use of metallic nanoparticles such as gold nanoparticles, in conjunction with benchtop XFCT, will be discussed within the context of preclinical multimodal multiplexed molecular imaging. Learning Objectives: To learn the basic principles of XRF/XFCT imaging To learn the latest advances in benchtop XFCT development for preclinical imaging Funding support received from NIH and DOD; Funding support received from GE Healthcare; Funding support received from Siemens AX; Patent royalties received from GE Healthcare; L. Wang, Funding Support: NIH; COI: Microphotoacoustics; S. Cho, Yes: ;NIH/NCI grant R01CA155446 DOD/PCRP grant W81XWH-12-1-0198.« less

WE-H-206-00: Advances in Preclinical Imaging

DOE Office of Scientific and Technical Information (OSTI.GOV)

NONE

Lihong V. Wang: Photoacoustic tomography (PAT), combining non-ionizing optical and ultrasonic waves via the photoacoustic effect, provides in vivo multiscale functional, metabolic, and molecular imaging. Broad applications include imaging of the breast, brain, skin, esophagus, colon, vascular system, and lymphatic system in humans or animals. Light offers rich contrast but does not penetrate biological tissue in straight paths as x-rays do. Consequently, high-resolution pure optical imaging (e.g., confocal microscopy, two-photon microscopy, and optical coherence tomography) is limited to penetration within the optical diffusion limit (∼1 mm in the skin). Ultrasonic imaging, on the contrary, provides fine spatial resolution but suffersmore » from both poor contrast in early-stage tumors and strong speckle artifacts. In PAT, pulsed laser light penetrates tissue and generates a small but rapid temperature rise, which induces emission of ultrasonic waves due to thermoelastic expansion. The ultrasonic waves, orders of magnitude less scattering than optical waves, are then detected to form high-resolution images of optical absorption at depths up to 7 cm, conquering the optical diffusion limit. PAT is the only modality capable of imaging across the length scales of organelles, cells, tissues, and organs (up to whole-body small animals) with consistent contrast. This rapidly growing technology promises to enable multiscale biological research and accelerate translation from microscopic laboratory discoveries to macroscopic clinical practice. PAT may also hold the key to label-free early detection of cancer by in vivo quantification of hypermetabolism, the quintessential hallmark of malignancy. Learning Objectives: To understand the contrast mechanism of PAT To understand the multiscale applications of PAT Benjamin M. W. Tsui: Multi-modality molecular imaging instrumentation and techniques have been major developments in small animal imaging that has contributed significantly to biomedical research during the past decade. The initial development was an extension of clinical PET/CT and SPECT/CT from human to small animals and combine the unique functional information obtained from PET and SPECT with anatomical information provided by the CT in registered multi-modality images. The requirements to image a mouse whose size is an order of magnitude smaller than that of a human have spurred advances in new radiation detector technologies, novel imaging system designs and special image reconstruction and processing techniques. Examples are new detector materials and designs with high intrinsic resolution, multi-pinhole (MPH) collimator design for much improved resolution and detection efficiency compared to the conventional collimator designs in SPECT, 3D high-resolution and artifact-free MPH and sparse-view image reconstruction techniques, and iterative image reconstruction methods with system response modeling for resolution recovery and image noise reduction for much improved image quality. The spatial resolution of PET and SPECT has improved from ∼6–12 mm to ∼1 mm a few years ago to sub-millimeter today. A recent commercial small animal SPECT system has achieved a resolution of ∼0.25 mm which surpasses that of a state-of-art PET system whose resolution is limited by the positron range. More recently, multimodality SA PET/MRI and SPECT/MRI systems have been developed in research laboratories. Also, multi-modality SA imaging systems that include other imaging modalities such as optical and ultrasound are being actively pursued. In this presentation, we will provide a review of the development, recent advances and future outlook of multi-modality molecular imaging of small animals. Learning Objectives: To learn about the two major multi-modality molecular imaging techniques of small animals. To learn about the spatial resolution achievable by the molecular imaging systems for small animal today. To learn about the new multi-modality imaging instrumentation and techniques that are being developed. Sang Hyun Cho; X-ray fluorescence (XRF) imaging, such as x-ray fluorescence computed tomography (XFCT), offers unique capabilities for accurate identification and quantification of metals within the imaging objects. As a result, it has emerged as a promising quantitative imaging modality in recent years, especially in conjunction with metal-based imaging probes. This talk will familiarize the audience with the basic principles of XRF/XFCT imaging. It will also cover the latest development of benchtop XFCT technology. Additionally, the use of metallic nanoparticles such as gold nanoparticles, in conjunction with benchtop XFCT, will be discussed within the context of preclinical multimodal multiplexed molecular imaging. Learning Objectives: To learn the basic principles of XRF/XFCT imaging To learn the latest advances in benchtop XFCT development for preclinical imaging Funding support received from NIH and DOD; Funding support received from GE Healthcare; Funding support received from Siemens AX; Patent royalties received from GE Healthcare; L. Wang, Funding Support: NIH; COI: Microphotoacoustics; S. Cho, Yes: ;NIH/NCI grant R01CA155446 DOD/PCRP grant W81XWH-12-1-0198.« less

Combined multimodal photoacoustic tomography, optical coherence tomography (OCT) and OCT based angiography system for in vivo imaging of multiple skin disorders in human(Conference Presentation)

NASA Astrophysics Data System (ADS)

Liu, Mengyang; Chen, Zhe; Sinz, Christoph; Rank, Elisabet; Zabihian, Behrooz; Zhang, Edward Z.; Beard, Paul C.; Kittler, Harald; Drexler, Wolfgang

2017-02-01

All optical photoacoustic tomography (PAT) using a planar Fabry-Perot interferometer polymer film sensor has been demonstrated for in vivo human palm imaging with an imaging penetration depth of 5 mm. The relatively larger vessels in the superficial plexus and the vessels in the dermal plexus are visible in PAT. However, due to both resolution and sensitivity limits, all optical PAT cannot reveal the smaller vessels such as capillary loops and venules. Melanin absorption also sometimes causes difficulties in PAT to resolve vessels. Optical coherence tomography (OCT) based angiography, on the other hand, has been proven suitable for microvasculature visualization in the first couple millimeters in human. In our work, we combine an all optical PAT system with an OCT system featuring a phase stable akinetic swept source. This multimodal PAT/OCT/OCT-angiography system provides us co-registered human skin vasculature information as well as the structural information of cutaneous. The scanning units of the sub-systems are assembled into one probe, which is then mounted onto a portable rack. The probe and rack design gives six degrees of freedom, allowing the multimodal optical imaging probe to access nearly all regions of human body. Utilizing this probe, we perform imaging on patients with various skin disorders as well as on healthy controls. Fused PAT/OCT-angiography volume shows the complete blood vessel network in human skin, which is further embedded in the morphology provided by OCT. A comparison between the results from the disordered regions and the normal regions demonstrates the clinical translational value of this multimodal optical imaging system in dermatology.

Multimodal full-field optical coherence tomography on biological tissue: toward all optical digital pathology

NASA Astrophysics Data System (ADS)

Harms, F.; Dalimier, E.; Vermeulen, P.; Fragola, A.; Boccara, A. C.

2012-03-01

Optical Coherence Tomography (OCT) is an efficient technique for in-depth optical biopsy of biological tissues, relying on interferometric selection of ballistic photons. Full-Field Optical Coherence Tomography (FF-OCT) is an alternative approach to Fourier-domain OCT (spectral or swept-source), allowing parallel acquisition of en-face optical sections. Using medium numerical aperture objective, it is possible to reach an isotropic resolution of about 1x1x1 ìm. After stitching a grid of acquired images, FF-OCT gives access to the architecture of the tissue, for both macroscopic and microscopic structures, in a non-invasive process, which makes the technique particularly suitable for applications in pathology. Here we report a multimodal approach to FF-OCT, combining two Full-Field techniques for collecting a backscattered endogeneous OCT image and a fluorescence exogeneous image in parallel. Considering pathological diagnosis of cancer, visualization of cell nuclei is of paramount importance. OCT images, even for the highest resolution, usually fail to identify individual nuclei due to the nature of the optical contrast used. We have built a multimodal optical microscope based on the combination of FF-OCT and Structured Illumination Microscopy (SIM). We used x30 immersion objectives, with a numerical aperture of 1.05, allowing for sub-micron transverse resolution. Fluorescent staining of nuclei was obtained using specific fluorescent dyes such as acridine orange. We present multimodal images of healthy and pathological skin tissue at various scales. This instrumental development paves the way for improvements of standard pathology procedures, as a faster, non sacrificial, operator independent digital optical method compared to frozen sections.

Voxel-based automated detection of focal cortical dysplasia lesions using diffusion tensor imaging and T2-weighted MRI data.

PubMed

Wang, Yanming; Zhou, Yawen; Wang, Huijuan; Cui, Jin; Nguchu, Benedictor Alexander; Zhang, Xufei; Qiu, Bensheng; Wang, Xiaoxiao; Zhu, Mingwang

2018-05-21

The aim of this study was to automatically detect focal cortical dysplasia (FCD) lesions in patients with extratemporal lobe epilepsy by relying on diffusion tensor imaging (DTI) and T2-weighted magnetic resonance imaging (MRI) data. We implemented an automated classifier using voxel-based multimodal features to identify gray and white matter abnormalities of FCD in patient cohorts. In addition to the commonly used T2-weighted image intensity feature, DTI-based features were also utilized. A Gaussian processes for machine learning (GPML) classifier was tested on 12 patients with FCD (8 with histologically confirmed FCD) scanned at 1.5 T and cross-validated using a leave-one-out strategy. Moreover, we compared the multimodal GPML paradigm's performance with that of single modal GPML and classical support vector machine (SVM). Our results demonstrated that the GPML performance on DTI-based features (mean AUC = 0.63) matches with the GPML performance on T2-weighted image intensity feature (mean AUC = 0.64). More promisingly, GPML yielded significantly improved performance (mean AUC = 0.76) when applying DTI-based features to multimodal paradigm. Based on the results, it can also be clearly stated that the proposed GPML strategy performed better and is robust to unbalanced dataset contrary to SVM that performed poorly (AUC = 0.69). Therefore, the GPML paradigm using multimodal MRI data containing DTI modality has promising result towards detection of the FCD lesions and provides an effective direction for future researches. Copyright © 2018 Elsevier Inc. All rights reserved.

Development of ClearPEM-Sonic, a multimodal mammography system for PET and Ultrasound

NASA Astrophysics Data System (ADS)

Cucciati, G.; Auffray, E.; Bugalho, R.; Cao, L.; Di Vara, N.; Farina, F.; Felix, N.; Frisch, B.; Ghezzi, A.; Juhan, V.; Jun, D.; Lasaygues, P.; Lecoq, P.; Mensah, S.; Mundler, O.; Neves, J.; Paganoni, M.; Peter, J.; Pizzichemi, M.; Siles, P.; Silva, J. C.; Silva, R.; Tavernier, S.; Tessonnier, L.; Varela, J.

2014-03-01

ClearPEM-Sonic is an innovative imaging device specifically developed for breast cancer. The possibility to work in PEM-Ultrasound multimodality allows to obtain metabolic and morphological information increasing the specificity of the exam. The ClearPEM detector is developed to maximize the sensitivity and the spatial resolution as compared to Whole-Body PET scanners. It is coupled with a 3D ultrasound system, the SuperSonic Imagine Aixplorer that improves the specificity of the exam by providing a tissue elasticity map. This work describes the ClearPEM-Sonic project focusing on the technological developments it has required, the technical merits (and limits) and the first multimodal images acquired on a dedicated phantom. It finally presents selected clinical case studies that confirm the value of PEM information.

Focusing and imaging with increased numerical apertures through multimode fibers with micro-fabricated optics.

PubMed

Bianchi, S; Rajamanickam, V P; Ferrara, L; Di Fabrizio, E; Liberale, C; Di Leonardo, R

2013-12-01

The use of individual multimode optical fibers in endoscopy applications has the potential to provide highly miniaturized and noninvasive probes for microscopy and optical micromanipulation. A few different strategies have been proposed recently, but they all suffer from intrinsically low resolution related to the low numerical aperture of multimode fibers. Here, we show that two-photon polymerization allows for direct fabrication of micro-optics components on the fiber end, resulting in an increase of the numerical aperture to a value that is close to 1. Coupling light into the fiber through a spatial light modulator, we were able to optically scan a submicrometer spot (300 nm FWHM) over an extended region, facing the opposite fiber end. Fluorescence imaging with improved resolution is also demonstrated.

Analysis of multimode fiber bundles for endoscopic spectral-domain optical coherence tomography

PubMed Central

Risi, Matthew D.; Makhlouf, Houssine; Rouse, Andrew R.; Gmitro, Arthur F.

2016-01-01

A theoretical analysis of the use of a fiber bundle in spectral-domain optical coherence tomography (OCT) systems is presented. The fiber bundle enables a flexible endoscopic design and provides fast, parallelized acquisition of the OCT data. However, the multimode characteristic of the fibers in the fiber bundle affects the depth sensitivity of the imaging system. A description of light interference in a multimode fiber is presented along with numerical simulations and experimental studies to illustrate the theoretical analysis. PMID:25967012

Children Creating Multimodal Stories about a Familiar Environment

ERIC Educational Resources Information Center

Kervin, Lisa; Mantei, Jessica

2017-01-01

Storytelling is a practice that enables children to apply their literacy skills. This article shares a collaborative literacy strategy devised to enable children to create multimodal stories about their familiar school environment. The strategy uses resources, including the children's own drawings, images from Google Maps, and the Puppet Pals…

Multimodal segmentation of optic disc and cup from stereo fundus and SD-OCT images

NASA Astrophysics Data System (ADS)

Miri, Mohammad Saleh; Lee, Kyungmoo; Niemeijer, Meindert; Abràmoff, Michael D.; Kwon, Young H.; Garvin, Mona K.

2013-03-01

Glaucoma is one of the major causes of blindness worldwide. One important structural parameter for the diagnosis and management of glaucoma is the cup-to-disc ratio (CDR), which tends to become larger as glaucoma progresses. While approaches exist for segmenting the optic disc and cup within fundus photographs, and more recently, within spectral-domain optical coherence tomography (SD-OCT) volumes, no approaches have been reported for the simultaneous segmentation of these structures within both modalities combined. In this work, a multimodal pixel-classification approach for the segmentation of the optic disc and cup within fundus photographs and SD-OCT volumes is presented. In particular, after segmentation of other important structures (such as the retinal layers and retinal blood vessels) and fundus-to-SD-OCT image registration, features are extracted from both modalities and a k-nearest-neighbor classification approach is used to classify each pixel as cup, rim, or background. The approach is evaluated on 70 multimodal image pairs from 35 subjects in a leave-10%-out fashion (by subject). A significant improvement in classification accuracy is obtained using the multimodal approach over that obtained from the corresponding unimodal approach (97.8% versus 95.2%; p < 0:05; paired t-test).

Mechanisms of murine cerebral malaria: Multimodal imaging of altered cerebral metabolism and protein oxidation at hemorrhage sites

PubMed Central

Hackett, Mark J.; Aitken, Jade B.; El-Assaad, Fatima; McQuillan, James A.; Carter, Elizabeth A.; Ball, Helen J.; Tobin, Mark J.; Paterson, David; de Jonge, Martin D.; Siegele, Rainer; Cohen, David D.; Vogt, Stefan; Grau, Georges E.; Hunt, Nicholas H.; Lay, Peter A.

2015-01-01

Using a multimodal biospectroscopic approach, we settle several long-standing controversies over the molecular mechanisms that lead to brain damage in cerebral malaria, which is a major health concern in developing countries because of high levels of mortality and permanent brain damage. Our results provide the first conclusive evidence that important components of the pathology of cerebral malaria include peroxidative stress and protein oxidation within cerebellar gray matter, which are colocalized with elevated nonheme iron at the site of microhemorrhage. Such information could not be obtained previously from routine imaging methods, such as electron microscopy, fluorescence, and optical microscopy in combination with immunocytochemistry, or from bulk assays, where the level of spatial information is restricted to the minimum size of tissue that can be dissected. We describe the novel combination of chemical probe–free, multimodal imaging to quantify molecular markers of disturbed energy metabolism and peroxidative stress, which were used to provide new insights into understanding the pathogenesis of cerebral malaria. In addition to these mechanistic insights, the approach described acts as a template for the future use of multimodal biospectroscopy for understanding the molecular processes involved in a range of clinically important acute and chronic (neurodegenerative) brain diseases to improve treatment strategies. PMID:26824064

A Deep and Autoregressive Approach for Topic Modeling of Multimodal Data.

PubMed

Zheng, Yin; Zhang, Yu-Jin; Larochelle, Hugo

2016-06-01

Topic modeling based on latent Dirichlet allocation (LDA) has been a framework of choice to deal with multimodal data, such as in image annotation tasks. Another popular approach to model the multimodal data is through deep neural networks, such as the deep Boltzmann machine (DBM). Recently, a new type of topic model called the Document Neural Autoregressive Distribution Estimator (DocNADE) was proposed and demonstrated state-of-the-art performance for text document modeling. In this work, we show how to successfully apply and extend this model to multimodal data, such as simultaneous image classification and annotation. First, we propose SupDocNADE, a supervised extension of DocNADE, that increases the discriminative power of the learned hidden topic features and show how to employ it to learn a joint representation from image visual words, annotation words and class label information. We test our model on the LabelMe and UIUC-Sports data sets and show that it compares favorably to other topic models. Second, we propose a deep extension of our model and provide an efficient way of training the deep model. Experimental results show that our deep model outperforms its shallow version and reaches state-of-the-art performance on the Multimedia Information Retrieval (MIR) Flickr data set.

Drug-related webpages classification based on multi-modal local decision fusion

NASA Astrophysics Data System (ADS)

Hu, Ruiguang; Su, Xiaojing; Liu, Yanxin

2018-03-01

In this paper, multi-modal local decision fusion is used for drug-related webpages classification. First, meaningful text are extracted through HTML parsing, and effective images are chosen by the FOCARSS algorithm. Second, six SVM classifiers are trained for six kinds of drug-taking instruments, which are represented by PHOG. One SVM classifier is trained for the cannabis, which is represented by the mid-feature of BOW model. For each instance in a webpage, seven SVMs give seven labels for its image, and other seven labels are given by searching the names of drug-taking instruments and cannabis in its related text. Concatenating seven labels of image and seven labels of text, the representation of those instances in webpages are generated. Last, Multi-Instance Learning is used to classify those drugrelated webpages. Experimental results demonstrate that the classification accuracy of multi-instance learning with multi-modal local decision fusion is much higher than those of single-modal classification.

Multimodal Deep Autoencoder for Human Pose Recovery.

PubMed

Hong, Chaoqun; Yu, Jun; Wan, Jian; Tao, Dacheng; Wang, Meng

2015-12-01

Video-based human pose recovery is usually conducted by retrieving relevant poses using image features. In the retrieving process, the mapping between 2D images and 3D poses is assumed to be linear in most of the traditional methods. However, their relationships are inherently non-linear, which limits recovery performance of these methods. In this paper, we propose a novel pose recovery method using non-linear mapping with multi-layered deep neural network. It is based on feature extraction with multimodal fusion and back-propagation deep learning. In multimodal fusion, we construct hypergraph Laplacian with low-rank representation. In this way, we obtain a unified feature description by standard eigen-decomposition of the hypergraph Laplacian matrix. In back-propagation deep learning, we learn a non-linear mapping from 2D images to 3D poses with parameter fine-tuning. The experimental results on three data sets show that the recovery error has been reduced by 20%-25%, which demonstrates the effectiveness of the proposed method.

Multi-modal diffuse optical techniques for breast cancer neoadjuvant chemotherapy monitoring (Conference Presentation)

NASA Astrophysics Data System (ADS)

Cochran, Jeffrey M.; Busch, David R.; Ban, Han Y.; Kavuri, Venkaiah C.; Schweiger, Martin J.; Arridge, Simon R.; Yodh, Arjun G.

2017-02-01

We present high spatial density, multi-modal, parallel-plate Diffuse Optical Tomography (DOT) imaging systems for the purpose of breast tumor detection. One hybrid instrument provides time domain (TD) and continuous wave (CW) DOT at 64 source fiber positions. The TD diffuse optical spectroscopy with PMT- detection produces low-resolution images of absolute tissue scattering and absorption while the spatially dense array of CCD-coupled detector fibers (108 detectors) provides higher-resolution CW images of relative tissue optical properties. Reconstruction of the tissue optical properties, along with total hemoglobin concentration and tissue oxygen saturation, is performed using the TOAST software suite. Comparison of the spatially-dense DOT images and MR images allows for a robust validation of DOT against an accepted clinical modality. Additionally, the structural information from co-registered MR images is used as a spatial prior to improve the quality of the functional optical images and provide more accurate quantification of the optical and hemodynamic properties of tumors. We also present an optical-only imaging system that provides frequency domain (FD) DOT at 209 source positions with full CCD detection and incorporates optical fringe projection profilometry to determine the breast boundary. This profilometry serves as a spatial constraint, improving the quality of the DOT reconstructions while retaining the benefits of an optical-only device. We present initial images from both human subjects and phantoms to display the utility of high spatial density data and multi-modal information in DOT reconstruction with the two systems.

Integration of Sparse Multi-modality Representation and Anatomical Constraint for Isointense Infant Brain MR Image Segmentation

PubMed Central

Wang, Li; Shi, Feng; Gao, Yaozong; Li, Gang; Gilmore, John H.; Lin, Weili; Shen, Dinggang

2014-01-01

Segmentation of infant brain MR images is challenging due to poor spatial resolution, severe partial volume effect, and the ongoing maturation and myelination process. During the first year of life, the brain image contrast between white and gray matters undergoes dramatic changes. In particular, the image contrast inverses around 6–8 months of age, where the white and gray matter tissues are isointense in T1 and T2 weighted images and hence exhibit the extremely low tissue contrast, posing significant challenges for automated segmentation. In this paper, we propose a general framework that adopts sparse representation to fuse the multi-modality image information and further incorporate the anatomical constraints for brain tissue segmentation. Specifically, we first derive an initial segmentation from a library of aligned images with ground-truth segmentations by using sparse representation in a patch-based fashion for the multi-modality T1, T2 and FA images. The segmentation result is further iteratively refined by integration of the anatomical constraint. The proposed method was evaluated on 22 infant brain MR images acquired at around 6 months of age by using a leave-one-out cross-validation, as well as other 10 unseen testing subjects. Our method achieved a high accuracy for the Dice ratios that measure the volume overlap between automated and manual segmentations, i.e., 0.889±0.008 for white matter and 0.870±0.006 for gray matter. PMID:24291615

Fluorescence labeled microbubbles for multimodal imaging.

PubMed

Barrefelt, Åsa; Zhao, Ying; Larsson, Malin K; Egri, Gabriella; Kuiper, Raoul V; Hamm, Jörg; Saghafian, Maryam; Caidahl, Kenneth; Brismar, Torkel B; Aspelin, Peter; Heuchel, Rainer; Muhammed, Mamoun; Dähne, Lars; Hassan, Moustapha

2015-08-28

Air-filled polyvinyl alcohol microbubbles (PVA-MBs) were recently introduced as a contrast agent for ultrasound imaging. In the present study, we explore the possibility of extending their application in multimodal imaging by labeling them with a near infrared (NIR) fluorophore, VivoTag-680. PVA-MBs were injected intravenously into FVB/N female mice and their dynamic biodistribution over 24 h was determined by 3D-fluorescence imaging co-registered with 3D-μCT imaging, to verify the anatomic location. To further confirm the biodistribution results from in vivo imaging, organs were removed and examined histologically using bright field and fluorescence microscopy. Fluorescence imaging detected PVA-MB accumulation in the lungs within the first 30 min post-injection. Redistribution to a low extent was observed in liver and kidneys at 4 h, and to a high extent mainly in the liver and spleen at 24 h. Histology confirmed PVA-MB localization in lung capillaries and macrophages. In the liver, they were associated with Kupffer cells; in the spleen, they were located mostly within the marginal-zone. Occasional MBs were observed in the kidney glomeruli and interstitium. The potential application of PVA-MBs as a contrast agent was also studied using ultrasound (US) imaging in subcutaneous and orthotopic pancreatic cancer mouse models, to visualize blood flow within the tumor mass. In conclusion, this study showed that PVA-MBs are useful as a contrast agent for multimodal imaging. Copyright © 2015 Elsevier Inc. All rights reserved.

Echocardiography in the Era of Multimodality Cardiovascular Imaging

PubMed Central

Shah, Benoy Nalin

2013-01-01

Echocardiography remains the most frequently performed cardiac imaging investigation and is an invaluable tool for detailed and accurate evaluation of cardiac structure and function. Echocardiography, nuclear cardiology, cardiac magnetic resonance imaging, and cardiovascular-computed tomography comprise the subspeciality of cardiovascular imaging, and these techniques are often used together for a multimodality, comprehensive assessment of a number of cardiac diseases. This paper provides the general cardiologist and physician with an overview of state-of-the-art modern echocardiography, summarising established indications as well as highlighting advances in stress echocardiography, three-dimensional echocardiography, deformation imaging, and contrast echocardiography. Strengths and limitations of echocardiography are discussed as well as the growing role of real-time three-dimensional echocardiography in the guidance of structural heart interventions in the cardiac catheter laboratory. PMID:23878804

Magnetic Nanoparticles for Multi-Imaging and Drug Delivery

PubMed Central

Lee, Jae-Hyun; Kim, Ji-wook; Cheon, Jinwoo

2013-01-01

Various bio-medical applications of magnetic nanoparticles have been explored during the past few decades. As tools that hold great potential for advancing biological sciences, magnetic nanoparticles have been used as platform materials for enhanced magnetic resonance imaging (MRI) agents, biological separation and magnetic drug delivery systems, and magnetic hyperthermia treatment. Furthermore, approaches that integrate various imaging and bioactive moieties have been used in the design of multi-modality systems, which possess synergistically enhanced properties such as better imaging resolution and sensitivity, molecular recognition capabilities, stimulus responsive drug delivery with on-demand control, and spatio-temporally controlled cell signal activation. Below, recent studies that focus on the design and synthesis of multi-mode magnetic nanoparticles will be briefly reviewed and their potential applications in the imaging and therapy areas will be also discussed. PMID:23579479

Rat brain imaging using full field optical coherence microscopy with short multimode fiber probe

NASA Astrophysics Data System (ADS)

Sato, Manabu; Saito, Daisuke; Kurotani, Reiko; Abe, Hiroyuki; Kawauchi, Satoko; Sato, Shunichi; Nishidate, Izumi

2017-02-01

We demonstrated FF OCM(full field optical coherence microscopy) using an ultrathin forward-imaging SMMF (short multimode fiber) probe of 50 μm core diameter, 125 μm diameter, and 7.4 mm length, which is a typical graded-index multimode fiber for optical communications. The axial resolution was measured to be 2.20 μm, which is close to the calculated axial resolution of 2.06 μm. The lateral resolution was evaluated to be 4.38 μm using a test pattern. Assuming that the FWHM of the contrast is the DOF (depth of focus), the DOF of the signal is obtained at 36 μm and that of the OCM is 66 μm. The contrast of the OCT images was 6.1 times higher than that of the signal images due to the coherence gate. After an euthanasia the rat brain was resected and cut at 2.6mm tail from Bregma. Contacting SMMF to the primary somatosensory cortex and the agranular insular cortex of ex vivo brain, OCM images of the brain were measured 100 times with 2μm step. 3D OCM images of the brain were measured, and internal structure information was obtained. The feasibility of an SMMF as an ultrathin forward-imaging probe in full-field OCM has been demonstrated.

Multimodality image display station

NASA Astrophysics Data System (ADS)

Myers, H. Joseph

1990-07-01

The Multi-modality Image Display Station (MIDS) is designed for the use of physicians outside of the radiology department. Connected to a local area network or a host computer, it provides speedy access to digitized radiology images and written diagnostics needed by attending and consulting physicians near the patient bedside. Emphasis has been placed on low cost, high performance and ease of use. The work is being done as a joint study with the University of Texas Southwestern Medical Center at Dallas, and as part of a joint development effort with the Mayo Clinic. MIDS is a prototype, and should not be assumed to be an IBM product.

Nanomaterials for In Vivo Imaging.

PubMed

Smith, Bryan Ronain; Gambhir, Sanjiv Sam

2017-02-08

In vivo imaging, which enables us to peer deeply within living subjects, is producing tremendous opportunities both for clinical diagnostics and as a research tool. Contrast material is often required to clearly visualize the functional architecture of physiological structures. Recent advances in nanomaterials are becoming pivotal to generate the high-resolution, high-contrast images needed for accurate, precision diagnostics. Nanomaterials are playing major roles in imaging by delivering large imaging payloads, yielding improved sensitivity, multiplexing capacity, and modularity of design. Indeed, for several imaging modalities, nanomaterials are now not simply ancillary contrast entities, but are instead the original and sole source of image signal that make possible the modality's existence. We address the physicochemical makeup/design of nanomaterials through the lens of the physical properties that produce contrast signal for the cognate imaging modality-we stratify nanomaterials on the basis of their (i) magnetic, (ii) optical, (iii) acoustic, and/or (iv) nuclear properties. We evaluate them for their ability to provide relevant information under preclinical and clinical circumstances, their in vivo safety profiles (which are being incorporated into their chemical design), their modularity in being fused to create multimodal nanomaterials (spanning multiple different physical imaging modalities and therapeutic/theranostic capabilities), their key properties, and critically their likelihood to be clinically translated.

Intrasubject multimodal groupwise registration with the conditional template entropy.

PubMed

Polfliet, Mathias; Klein, Stefan; Huizinga, Wyke; Paulides, Margarethus M; Niessen, Wiro J; Vandemeulebroucke, Jef

2018-05-01

Image registration is an important task in medical image analysis. Whereas most methods are designed for the registration of two images (pairwise registration), there is an increasing interest in simultaneously aligning more than two images using groupwise registration. Multimodal registration in a groupwise setting remains difficult, due to the lack of generally applicable similarity metrics. In this work, a novel similarity metric for such groupwise registration problems is proposed. The metric calculates the sum of the conditional entropy between each image in the group and a representative template image constructed iteratively using principal component analysis. The proposed metric is validated in extensive experiments on synthetic and intrasubject clinical image data. These experiments showed equivalent or improved registration accuracy compared to other state-of-the-art (dis)similarity metrics and improved transformation consistency compared to pairwise mutual information. Copyright © 2018 The Authors. Published by Elsevier B.V. All rights reserved.

BNU-LSVED: a multimodal spontaneous expression database in educational environment

NASA Astrophysics Data System (ADS)

Sun, Bo; Wei, Qinglan; He, Jun; Yu, Lejun; Zhu, Xiaoming

2016-09-01

In the field of pedagogy or educational psychology, emotions are treated as very important factors, which are closely associated with cognitive processes. Hence, it is meaningful for teachers to analyze students' emotions in classrooms, thus adjusting their teaching activities and improving students ' individual development. To provide a benchmark for different expression recognition algorithms, a large collection of training and test data in classroom environment has become an acute problem that needs to be resolved. In this paper, we present a multimodal spontaneous database in real learning environment. To collect the data, students watched seven kinds of teaching videos and were simultaneously filmed by a camera. Trained coders made one of the five learning expression labels for each image sequence extracted from the captured videos. This subset consists of 554 multimodal spontaneous expression image sequences (22,160 frames) recorded in real classrooms. There are four main advantages in this database. 1) Due to recorded in the real classroom environment, viewer's distance from the camera and the lighting of the database varies considerably between image sequences. 2) All the data presented are natural spontaneous responses to teaching videos. 3) The multimodal database also contains nonverbal behavior including eye movement, head posture and gestures to infer a student ' s affective state during the courses. 4) In the video sequences, there are different kinds of temporal activation patterns. In addition, we have demonstrated the labels for the image sequences are in high reliability through Cronbach's alpha method.

New developments in multimodal clinical multiphoton tomography

NASA Astrophysics Data System (ADS)

König, Karsten

2011-03-01

80 years ago, the PhD student Maria Goeppert predicted in her thesis in Goettingen, Germany, two-photon effects. It took 30 years to prove her theory, and another three decades to realize the first two-photon microscope. With the beginning of this millennium, first clinical multiphoton tomographs started operation in research institutions, hospitals, and in the cosmetic industry. The multiphoton tomograph MPTflexTM with its miniaturized flexible scan head became the Prism-Award 2010 winner in the category Life Sciences. Multiphoton tomographs with its superior submicron spatial resolution can be upgraded to 5D imaging tools by adding spectral time-correlated single photon counting units. Furthermore, multimodal hybrid tomographs provide chemical fingerprinting and fast wide-field imaging. The world's first clinical CARS studies have been performed with a hybrid multimodal multiphoton tomograph in spring 2010. In particular, nonfluorescent lipids and water as well as mitochondrial fluorescent NAD(P)H, fluorescent elastin, keratin, and melanin as well as SHG-active collagen have been imaged in patients with dermatological disorders. Further multimodal approaches include the combination of multiphoton tomographs with low-resolution imaging tools such as ultrasound, optoacoustic, OCT, and dermoscopy systems. Multiphoton tomographs are currently employed in Australia, Japan, the US, and in several European countries for early diagnosis of skin cancer (malignant melanoma), optimization of treatment strategies (wound healing, dermatitis), and cosmetic research including long-term biosafety tests of ZnO sunscreen nanoparticles and the measurement of the stimulated biosynthesis of collagen by anti-ageing products.

Multimodal nanoparticle imaging agents: design and applications

NASA Astrophysics Data System (ADS)

Burke, Benjamin P.; Cawthorne, Christopher; Archibald, Stephen J.

2017-10-01

Molecular imaging, where the location of molecules or nanoscale constructs can be tracked in the body to report on disease or biochemical processes, is rapidly expanding to include combined modality or multimodal imaging. No single imaging technique can offer the optimum combination of properties (e.g. resolution, sensitivity, cost, availability). The rapid technological advances in hardware to scan patients, and software to process and fuse images, are pushing the boundaries of novel medical imaging approaches, and hand-in-hand with this is the requirement for advanced and specific multimodal imaging agents. These agents can be detected using a selection from radioisotope, magnetic resonance and optical imaging, among others. Nanoparticles offer great scope in this area as they lend themselves, via facile modification procedures, to act as multifunctional constructs. They have relevance as therapeutics and drug delivery agents that can be tracked by molecular imaging techniques with the particular development of applications in optically guided surgery and as radiosensitizers. There has been a huge amount of research work to produce nanoconstructs for imaging, and the parameters for successful clinical translation and validation of therapeutic applications are now becoming much better understood. It is an exciting time of progress for these agents as their potential is closer to being realized with translation into the clinic. The coming 5-10 years will be critical, as we will see if the predicted improvement in clinical outcomes becomes a reality. Some of the latest advances in combination modality agents are selected and the progression pathway to clinical trials analysed. This article is part of the themed issue 'Challenges for chemistry in molecular imaging'.

Multimodality hard-x-ray imaging of a chromosome with nanoscale spatial resolution

DOE PAGES

Yan, Hanfei; Nazaretski, Evgeny; Lauer, Kenneth R.; ...

2016-02-05

Here, we developed a scanning hard x-ray microscope using a new class of x-ray nano-focusing optic called a multilayer Laue lens and imaged a chromosome with nanoscale spatial resolution. The combination of the hard x-ray's superior penetration power, high sensitivity to elemental composition, high spatial-resolution and quantitative analysis creates a unique tool with capabilities that other microscopy techniques cannot provide. Using this microscope, we simultaneously obtained absorption-, phase-, and fluorescence-contrast images of Pt-stained human chromosome samples. The high spatial-resolution of the microscope and its multi-modality imaging capabilities enabled us to observe the internal ultra-structures of a thick chromosome without sectioningmore » it.« less

Multimodality hard-x-ray imaging of a chromosome with nanoscale spatial resolution

DOE Office of Scientific and Technical Information (OSTI.GOV)

Yan, Hanfei; Nazaretski, Evgeny; Lauer, Kenneth R.

Here, we developed a scanning hard x-ray microscope using a new class of x-ray nano-focusing optic called a multilayer Laue lens and imaged a chromosome with nanoscale spatial resolution. The combination of the hard x-ray's superior penetration power, high sensitivity to elemental composition, high spatial-resolution and quantitative analysis creates a unique tool with capabilities that other microscopy techniques cannot provide. Using this microscope, we simultaneously obtained absorption-, phase-, and fluorescence-contrast images of Pt-stained human chromosome samples. The high spatial-resolution of the microscope and its multi-modality imaging capabilities enabled us to observe the internal ultra-structures of a thick chromosome without sectioningmore » it.« less

A Multimode Optical Imaging System for Preclinical Applications In Vivo: Technology Development, Multiscale Imaging, and Chemotherapy Assessment

PubMed Central

Hwang, Jae Youn; Wachsmann-Hogiu, Sebastian; Ramanujan, V. Krishnan; Ljubimova, Julia; Gross, Zeev; Gray, Harry B.; Medina-Kauwe, Lali K.; Farkas, Daniel L.

2012-01-01

Purpose Several established optical imaging approaches have been applied, usually in isolation, to preclinical studies; however, truly useful in vivo imaging may require a simultaneous combination of imaging modalities to examine dynamic characteristics of cells and tissues. We developed a new multimode optical imaging system designed to be application-versatile, yielding high sensitivity, and specificity molecular imaging. Procedures We integrated several optical imaging technologies, including fluorescence intensity, spectral, lifetime, intravital confocal, two-photon excitation, and bioluminescence, into a single system that enables functional multiscale imaging in animal models. Results The approach offers a comprehensive imaging platform for kinetic, quantitative, and environmental analysis of highly relevant information, with micro-to-macroscopic resolution. Applied to small animals in vivo, this provides superior monitoring of processes of interest, represented here by chemo-/nanoconstruct therapy assessment. Conclusions This new system is versatile and can be optimized for various applications, of which cancer detection and targeted treatment are emphasized here. PMID:21874388

Cross-modal face recognition using multi-matcher face scores

NASA Astrophysics Data System (ADS)

Zheng, Yufeng; Blasch, Erik

2015-05-01

The performance of face recognition can be improved using information fusion of multimodal images and/or multiple algorithms. When multimodal face images are available, cross-modal recognition is meaningful for security and surveillance applications. For example, a probe face is a thermal image (especially at nighttime), while only visible face images are available in the gallery database. Matching a thermal probe face onto the visible gallery faces requires crossmodal matching approaches. A few such studies were implemented in facial feature space with medium recognition performance. In this paper, we propose a cross-modal recognition approach, where multimodal faces are cross-matched in feature space and the recognition performance is enhanced with stereo fusion at image, feature and/or score level. In the proposed scenario, there are two cameras for stereo imaging, two face imagers (visible and thermal images) in each camera, and three recognition algorithms (circular Gaussian filter, face pattern byte, linear discriminant analysis). A score vector is formed with three cross-matched face scores from the aforementioned three algorithms. A classifier (e.g., k-nearest neighbor, support vector machine, binomial logical regression [BLR]) is trained then tested with the score vectors by using 10-fold cross validations. The proposed approach was validated with a multispectral stereo face dataset from 105 subjects. Our experiments show very promising results: ACR (accuracy rate) = 97.84%, FAR (false accept rate) = 0.84% when cross-matching the fused thermal faces onto the fused visible faces by using three face scores and the BLR classifier.

Multimodal Reading Comprehension: Curriculum Expectations and Large-Scale Literacy Testing Practices

ERIC Educational Resources Information Center

Unsworth, Len

2014-01-01

Interpreting the image-language interface in multimodal texts is now well recognized as a crucial aspect of reading comprehension in a number of official school syllabi such as the recently published Australian Curriculum: English (ACE). This article outlines the relevant expected student learning outcomes in this curriculum and draws attention to…

An fMRI Study of Multimodal Semantic and Phonological Processing in Reading Disabled Adolescents

ERIC Educational Resources Information Center

Landi, Nicole; Mencl, W. Einar; Frost, Stephen J.; Sandak, Rebecca; Pugh, Kenneth R.

2010-01-01

Using functional magnetic resonance imaging, we investigated multimodal (visual and auditory) semantic and unimodal (visual only) phonological processing in reading disabled (RD) adolescents and non-impaired (NI) control participants. We found reduced activation for RD relative to NI in a number of left-hemisphere reading-related areas across all…

A Multimodal Perspective on Textuality and Contexts

ERIC Educational Resources Information Center

Jewitt, Carey

2007-01-01

Textuality is often thought of in linguistic terms; for instance, the talk and writing that circulate in the classroom. In this paper I take a multimodal perspective on textuality and context. I draw on illustrative examples from school Science and English to examine how image, colour, gesture, gaze, posture and movement--as well as writing and…

"Convince Me!" Valuing Multimodal Literacies and Composing Public Service Announcements

ERIC Educational Resources Information Center

Selfe, Richard J.; Selfe, Cynthia L.

2008-01-01

For some teachers, the increasing attention to digital and multimodal composing in English and Language Arts classrooms has brought into sharp relief the profession's investment in print as the primary means of expression. Although new forms of communication that combine words, still and moving images, and animation have begun to dominate digital…

Concept for Classifying Facade Elements Based on Material, Geometry and Thermal Radiation Using Multimodal Uav Remote Sensing

NASA Astrophysics Data System (ADS)

Ilehag, R.; Schenk, A.; Hinz, S.

2017-08-01

This paper presents a concept for classification of facade elements, based on the material and the geometry of the elements in addition to the thermal radiation of the facade with the usage of a multimodal Unmanned Aerial Vehicle (UAV) system. Once the concept is finalized and functional, the workflow can be used for energy demand estimations for buildings by exploiting existing methods for estimation of heat transfer coefficient and the transmitted heat loss. The multimodal system consists of a thermal, a hyperspectral and an optical sensor, which can be operational with a UAV. While dealing with sensors that operate in different spectra and have different technical specifications, such as the radiometric and the geometric resolution, the challenges that are faced are presented. Addressed are the different approaches of data fusion, such as image registration, generation of 3D models by performing image matching and the means for classification based on either the geometry of the object or the pixel values. As a first step towards realizing the concept, the result from a geometric calibration with a designed multimodal calibration pattern is presented.

MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization

PubMed Central

Mangin, Olivier; Filliat, David; ten Bosch, Louis; Oudeyer, Pierre-Yves

2015-01-01

In this paper we introduce MCA-NMF, a computational model of the acquisition of multimodal concepts by an agent grounded in its environment. More precisely our model finds patterns in multimodal sensor input that characterize associations across modalities (speech utterances, images and motion). We propose this computational model as an answer to the question of how some class of concepts can be learnt. In addition, the model provides a way of defining such a class of plausibly learnable concepts. We detail why the multimodal nature of perception is essential to reduce the ambiguity of learnt concepts as well as to communicate about them through speech. We then present a set of experiments that demonstrate the learning of such concepts from real non-symbolic data consisting of speech sounds, images, and motions. Finally we consider structure in perceptual signals and demonstrate that a detailed knowledge of this structure, named compositional understanding can emerge from, instead of being a prerequisite of, global understanding. An open-source implementation of the MCA-NMF learner as well as scripts and associated experimental data to reproduce the experiments are publicly available. PMID:26489021

Design and characterization of a handheld multimodal imaging device for the assessment of oral epithelial lesions

NASA Astrophysics Data System (ADS)

Higgins, Laura M.; Pierce, Mark C.

2014-08-01

A compact handpiece combining high resolution fluorescence (HRF) imaging with optical coherence tomography (OCT) was developed to provide real-time assessment of oral lesions. This multimodal imaging device simultaneously captures coregistered en face images with subcellular detail alongside cross-sectional images of tissue microstructure. The HRF imaging acquires a 712×594 μm2 field-of-view at the sample with a spatial resolution of 3.5 μm. The OCT images were acquired to a depth of 1.5 mm with axial and lateral resolutions of 9.3 and 8.0 μm, respectively. HRF and OCT images are simultaneously displayed at 25 fps. The handheld device was used to image a healthy volunteer, demonstrating the potential for in vivo assessment of the epithelial surface for dysplastic and neoplastic changes at the cellular level, while simultaneously evaluating submucosal involvement. We anticipate potential applications in real-time assessment of oral lesions for improved surveillance and surgical guidance.

NOVEL PRERETINAL HAIR PIN-LIKE VESSEL IN RETINAL ASTROCYTIC HAMARTOMA WITH VITREOUS HEMORRHAGE.

PubMed

Soeta, Megumi; Arai, Yusuke; Takahashi, Hidenori; Fujino, Yujiro; Tanabe, Tatsuro; Inoue, Yuji; Kawashima, Hidetoshi

2018-01-01

To report a case of retinal astrocytic hamartoma with vitreous hemorrhage and a hair pin-like vessel adhering to a posterior vitreous membrane. A 33-year-old man with a retinal astrocytic hamartoma presented with vitreous hemorrhage 5 times. Multimodal imaging, including fundus photography, fluorescein angiography, optical coherence tomography, and B-mode ultrasonography. Multimodal imaging demonstrated a novel hair pin-like vessel that adhered to the posterior vitreous membrane. Some cases of retinal astrocytic hamartoma with vitreous hemorrhage may be related to structure abnormalities of tumor vessels.

Papillary fibroelastoma diagnosed through multimodality cardiac imaging: a rare tumour in an uncommon location with review of literature.

PubMed

Anand, Senthil; Sydow, Nicole; Janardhanan, Rajesh

2017-08-08

We describe the case of a woman presenting with transient ischaemic attack, who was found to have a papillary fibroelastoma arising from the aortic wall, an extremely rare location. We describe the multimodality imaging techniques used in diagnosing this patient and review the most recent literature on evaluation and management of patients with cardiac papillary fibroelastomas. © BMJ Publishing Group Ltd (unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Hybrid Core-Shell (HyCoS) Nanoparticles produced by Complex Coacervation for Multimodal Applications

NASA Astrophysics Data System (ADS)

Vecchione, D.; Grimaldi, A. M.; Forte, E.; Bevilacqua, Paolo; Netti, P. A.; Torino, E.

2017-03-01

Multimodal imaging probes can provide diagnostic information combining different imaging modalities. Nanoparticles (NPs) can contain two or more imaging tracers that allow several diagnostic techniques to be used simultaneously. In this work, a complex coacervation process to produce core-shell completely biocompatible polymeric nanoparticles (HyCoS) for multimodal imaging applications is described. Innovations on the traditional coacervation process are found in the control of the reaction temperature, allowing a speeding up of the reaction itself, and the production of a double-crosslinked system to improve the stability of the nanostructures in the presence of a clinically relevant contrast agent for MRI (Gd-DTPA). Through the control of the crosslinking behavior, an increase up to 6 times of the relaxometric properties of the Gd-DTPA is achieved. Furthermore, HyCoS can be loaded with a high amount of dye such as ATTO 633 or conjugated with a model dye such as FITC for in vivo optical imaging. The results show stable core-shell polymeric nanoparticles that can be used both for MRI and for optical applications allowing detection free from harmful radiation. Additionally, preliminary results about the possibility to trigger the release of a drug through a pH effect are reported.

Exploring photoreceptor reflectivity via multimodal imaging of outer retinal tubulation in advanced age-related macular degeneration

PubMed Central

Litts, Katie M.; Wang, Xiaolin; Clark, Mark E.; Owsley, Cynthia; Freund, K. Bailey; Curcio, Christine A.; Zhang, Yuhua

2016-01-01

Purpose To investigate the microscopic structure of outer retinal tubulation (ORT) and optical properties of cone photoreceptors in vivo, we studied ORT appearance by multimodal imaging, including spectral domain optical coherence tomography (SD-OCT) and adaptive optics scanning laser ophthalmoscopy (AOSLO). Methods Four eyes of 4 subjects with advanced AMD underwent color fundus photography, infrared reflectance imaging, SD-OCT, and AOSLO with a high-resolution research instrument. ORT was identified in closely spaced (11 μm) SD-OCT volume scans. Results ORT in cross-sectional and en face SD-OCT was a hyporeflective area representing a lumen surrounded by a hyperreflective border consisting of cone photoreceptor mitochondria and external limiting membrane, per previous histology. In contrast, ORT by AOSLO was a hyporeflective structure of the same shape as in en face SD-OCT but lacking visualizable cone photoreceptors. Conclusion Lack of ORT cone reflectivity by AOSLO indicates that cones have lost their normal directionality and waveguiding property due to loss of outer segments and subsequent retinal remodeling. Reflective ORT cones by SD-OCT, in contrast, may depend partly on mitochondria as light scatterers within inner segments of these degenerating cells, a phenomenon enhanced by coherent imaging. Multimodal imaging of ORT provides insight into cone degeneration and reflectivity sources in OCT. PMID:27584549

The Neurochemical and Microstructural Changes in the Brain of Systemic Lupus Erythematosus Patients: A Multimodal MRI Study

PubMed Central

Zhang, Zhiyan; Wang, Yukai; Shen, Zhiwei; Yang, Zhongxian; Li, Li; Chen, Dongxiao; Yan, Gen; Cheng, Xiaofang; Shen, Yuanyu; Tang, Xiangyong; Hu, Wei; Wu, Renhua

2016-01-01

The diagnosis and pathology of neuropsychiatric systemic lupus erythematosus (NPSLE) remains challenging. Herein, we used multimodal imaging to assess anatomical and functional changes in brains of SLE patients instead of a single MRI approach generally used in previous studies. Twenty-two NPSLE patients, 21 non-NPSLE patients and 20 healthy controls (HCs) underwent 3.0 T MRI with multivoxel magnetic resonance spectroscopy, T1-weighted volumetric images for voxel based morphometry (VBM) and diffusional kurtosis imaging (DKI) scans. While there were findings in other basal ganglia regions, the most consistent findings were observed in the posterior cingulate gyrus (PCG). The reduction of multiple metabolite concentration was observed in the PCG in the two patient groups, and the NPSLE patients were more prominent. The two patient groups displayed lower diffusional kurtosis (MK) values in the bilateral PCG compared with HCs (p < 0.01) as assessed by DKI. Grey matter reduction in the PCG was observed in the NPSLE group using VBM. Positive correlations among cognitive function scores and imaging metrics in bilateral PCG were detected. Multimodal imaging is useful for evaluating SLE subjects and potentially determining disease pathology. Impairments of cognitive function in SLE patients may be interpreted by metabolic and microstructural changes in the PCG. PMID:26758023

Luminomagnetic Eu3+- and Dy3+-doped hydroxyapatite for multimodal imaging.

PubMed

Tesch, Annemarie; Wenisch, Christoph; Herrmann, Karl-Heinz; Reichenbach, Jürgen R; Warncke, Paul; Fischer, Dagmar; Müller, Frank A

2017-12-01

Multimodal imaging has recently attracted much attention due to the advantageous combination of different imaging modalities, like photoluminescence (PL) and magnetic resonance imaging (MRI). In the present study, luminescent and magnetic hydroxyapatites (HAp) were prepared via doping with europium (Eu 3+ ) and dysprosium (Dy 3+ ), respectively. Co-doping of Eu 3+ and Dy 3+ was used to combine the desired physical properties. Both lanthanide ions were successfully incorporated in the HAp crystal lattice, where they preferentially occupied calcium(I) sites. While Eu-doped HAp (Eu:HAp) exhibits dopant concentration dependent persistent PL properties, Dy-doped HAp (Dy:HAp) shows paramagnetic behavior due to the high magnetic moment of Dy 3+ . Co-doped HAp (Eu:Dy:HAp) nanoparticles combine both properties in one single crystal. Remarkably, multimodal co-doped HAp features enhanced PL properties due to an energy transfer from Dy 3+ sensitizer to Eu 3+ activator ions. Eu:Dy:HAp exhibits strong transverse relaxation effects with a maximum transverse relaxivity of 83.3L/(mmol·s). Due to their tunable PL, magnetic properties and cytocompatibility Eu:-, Dy:- and Eu:Dy:HAp represent promising biocompatible ceramic materials for luminescence imaging that simultaneously may serve as a contrast agent for MRI in permanent implants or functional coatings. Copyright © 2017 Elsevier B.V. All rights reserved.

A new approach of building 3D visualization framework for multimodal medical images display and computed assisted diagnosis

NASA Astrophysics Data System (ADS)

Li, Zhenwei; Sun, Jianyong; Zhang, Jianguo

2012-02-01

As more and more CT/MR studies are scanning with larger volume of data sets, more and more radiologists and clinician would like using PACS WS to display and manipulate these larger data sets of images with 3D rendering features. In this paper, we proposed a design method and implantation strategy to develop 3D image display component not only with normal 3D display functions but also with multi-modal medical image fusion as well as compute-assisted diagnosis of coronary heart diseases. The 3D component has been integrated into the PACS display workstation of Shanghai Huadong Hospital, and the clinical practice showed that it is easy for radiologists and physicians to use these 3D functions such as multi-modalities' (e.g. CT, MRI, PET, SPECT) visualization, registration and fusion, and the lesion quantitative measurements. The users were satisfying with the rendering speeds and quality of 3D reconstruction. The advantages of the component include low requirements for computer hardware, easy integration, reliable performance and comfortable application experience. With this system, the radiologists and the clinicians can manipulate with 3D images easily, and use the advanced visualization tools to facilitate their work with a PACS display workstation at any time.

Shape-Controlled Synthesis of Isotopic Yttrium-90-Labeled Rare Earth Fluoride Nanocrystals for Multimodal Imaging.

PubMed

Paik, Taejong; Chacko, Ann-Marie; Mikitsh, John L; Friedberg, Joseph S; Pryma, Daniel A; Murray, Christopher B

2015-09-22

Isotopically labeled nanomaterials have recently attracted much attention in biomedical research, environmental health studies, and clinical medicine because radioactive probes allow the elucidation of in vitro and in vivo cellular transport mechanisms, as well as the unambiguous distribution and localization of nanomaterials in vivo. In addition, nanocrystal-based inorganic materials have a unique capability of customizing size, shape, and composition; with the potential to be designed as multimodal imaging probes. Size and shape of nanocrystals can directly influence interactions with biological systems, hence it is important to develop synthetic methods to design radiolabeled nanocrystals with precise control of size and shape. Here, we report size- and shape-controlled synthesis of rare earth fluoride nanocrystals doped with the β-emitting radioisotope yttrium-90 ((90)Y). Size and shape of nanocrystals are tailored via tight control of reaction parameters and the type of rare earth hosts (e.g., Gd or Y) employed. Radiolabeled nanocrystals are synthesized in high radiochemical yield and purity as well as excellent radiolabel stability in the face of surface modification with different polymeric ligands. We demonstrate the Cerenkov radioluminescence imaging and magnetic resonance imaging capabilities of (90)Y-doped GdF3 nanoplates, which offer unique opportunities as a promising platform for multimodal imaging and targeted therapy.

Development of a Multi-modal Tissue Diagnostic System Combining High Frequency Ultrasound and Photoacoustic Imaging with Lifetime Fluorescence Spectroscopy

PubMed Central

Sun, Yang; Stephens, Douglas N.; Park, Jesung; Sun, Yinghua; Marcu, Laura; Cannata, Jonathan M.; Shung, K. Kirk

2010-01-01

We report the development and validate a multi-modal tissue diagnostic technology, which combines three complementary techniques into one system including ultrasound backscatter microscopy (UBM), photoacoustic imaging (PAI), and time-resolved laser-induced fluorescence spectroscopy (TR-LIFS). UBM enables the reconstruction of the tissue microanatomy. PAI maps the optical absorption heterogeneity of the tissue associated with structure information and has the potential to provide functional imaging of the tissue. Examination of the UBM and PAI images allows for localization of regions of interest for TR-LIFS evaluation of the tissue composition. The hybrid probe consists of a single element ring transducer with concentric fiber optics for multi-modal data acquisition. Validation and characterization of the multi-modal system and ultrasonic, photoacoustic, and spectroscopic data coregistration were conducted in a physical phantom with properties of ultrasound scattering, optical absorption, and fluorescence. The UBM system with the 41 MHz ring transducer can reach the axial and lateral resolution of 30 and 65 μm, respectively. The PAI system with 532 nm excitation light from a Nd:YAG laser shows great contrast for the distribution of optical absorbers. The TR-LIFS system records the fluorescence decay with the time resolution of ~300 ps and a high sensitivity of nM concentration range. Biological phantom constructed with different types of tissues (tendon and fat) was used to demonstrate the complementary information provided by the three modalities. Fluorescence spectra and lifetimes were compared to differentiate chemical composition of tissues at the regions of interest determined by the coregistered high resolution UBM and PAI image. Current results demonstrate that the fusion of these techniques enables sequentially detection of functional, morphological, and compositional features of biological tissue, suggesting potential applications in diagnosis of tumors and atherosclerotic plaques. PMID:21894259

Development of a Multi-modal Tissue Diagnostic System Combining High Frequency Ultrasound and Photoacoustic Imaging with Lifetime Fluorescence Spectroscopy.

PubMed

Sun, Yang; Stephens, Douglas N; Park, Jesung; Sun, Yinghua; Marcu, Laura; Cannata, Jonathan M; Shung, K Kirk

2008-01-01

We report the development and validate a multi-modal tissue diagnostic technology, which combines three complementary techniques into one system including ultrasound backscatter microscopy (UBM), photoacoustic imaging (PAI), and time-resolved laser-induced fluorescence spectroscopy (TR-LIFS). UBM enables the reconstruction of the tissue microanatomy. PAI maps the optical absorption heterogeneity of the tissue associated with structure information and has the potential to provide functional imaging of the tissue. Examination of the UBM and PAI images allows for localization of regions of interest for TR-LIFS evaluation of the tissue composition. The hybrid probe consists of a single element ring transducer with concentric fiber optics for multi-modal data acquisition. Validation and characterization of the multi-modal system and ultrasonic, photoacoustic, and spectroscopic data coregistration were conducted in a physical phantom with properties of ultrasound scattering, optical absorption, and fluorescence. The UBM system with the 41 MHz ring transducer can reach the axial and lateral resolution of 30 and 65 μm, respectively. The PAI system with 532 nm excitation light from a Nd:YAG laser shows great contrast for the distribution of optical absorbers. The TR-LIFS system records the fluorescence decay with the time resolution of ~300 ps and a high sensitivity of nM concentration range. Biological phantom constructed with different types of tissues (tendon and fat) was used to demonstrate the complementary information provided by the three modalities. Fluorescence spectra and lifetimes were compared to differentiate chemical composition of tissues at the regions of interest determined by the coregistered high resolution UBM and PAI image. Current results demonstrate that the fusion of these techniques enables sequentially detection of functional, morphological, and compositional features of biological tissue, suggesting potential applications in diagnosis of tumors and atherosclerotic plaques.

IDH mutation assessment of glioma using texture features of multimodal MR images

NASA Astrophysics Data System (ADS)

Zhang, Xi; Tian, Qiang; Wu, Yu-Xia; Xu, Xiao-Pan; Li, Bao-Juan; Liu, Yi-Xiong; Liu, Yang; Lu, Hong-Bing

2017-03-01

Purpose: To 1) find effective texture features from multimodal MRI that can distinguish IDH mutant and wild status, and 2) propose a radiomic strategy for preoperatively detecting IDH mutation patients with glioma. Materials and Methods: 152 patients with glioma were retrospectively included from the Cancer Genome Atlas. Corresponding T1-weighted image before- and post-contrast, T2-weighted image and fluid-attenuation inversion recovery image from the Cancer Imaging Archive were analyzed. Specific statistical tests were applied to analyze the different kind of baseline information of LrGG patients. Finally, 168 texture features were derived from multimodal MRI per patient. Then the support vector machine-based recursive feature elimination (SVM-RFE) and classification strategy was adopted to find the optimal feature subset and build the identification models for detecting the IDH mutation. Results: Among 152 patients, 92 and 60 were confirmed to be IDH-wild and mutant, respectively. Statistical analysis showed that the patients without IDH mutation was significant older than patients with IDH mutation (p<0.01), and the distribution of some histological subtypes was significant different between IDH wild and mutant groups (p<0.01). After SVM-RFE, 15 optimal features were determined for IDH mutation detection. The accuracy, sensitivity, specificity, and AUC after SVM-RFE and parameter optimization were 82.2%, 85.0%, 78.3%, and 0.841, respectively. Conclusion: This study presented a radiomic strategy for noninvasively discriminating IDH mutation of patients with glioma. It effectively incorporated kinds of texture features from multimodal MRI, and SVM-based classification strategy. Results suggested that features selected from SVM-RFE were more potential to identifying IDH mutation. The proposed radiomics strategy could facilitate the clinical decision making in patients with glioma.

Frameless multimodal image guidance of localized convection-enhanced delivery of therapeutics in the brain

PubMed Central

van der Bom, Imramsjah M J; Moser, Richard P; Gao, Guanping; Sena-Esteves, Miguel; Aronin, Neil

2013-01-01

Introduction Convection-enhanced delivery (CED) has been shown to be an effective method of administering macromolecular compounds into the brain that are unable to cross the blood-brain barrier. Because the administration is highly localized, accurate cannula placement by minimally invasive surgery is an important requisite. This paper reports on the use of an angiographic c-arm system which enables truly frameless multimodal image guidance during CED surgery. Methods A microcannula was placed into the striatum of five sheep under real-time fluoroscopic guidance using imaging data previously acquired by cone beam computed tomography (CBCT) and MRI, enabling three-dimensional navigation. After introduction of the cannula, high resolution CBCT was performed and registered with MRI to confirm the position of the cannula tip and to make adjustments as necessary. Adeno-associated viral vector-10, designed to deliver small-hairpin micro RNA (shRNAmir), was mixed with 2.0 mM gadolinium (Gd) and infused at a rate of 3 μl/min for a total of 100 μl. Upon completion, the animals were transferred to an MR scanner to assess the approximate distribution by measuring the volume of spread of Gd. Results The cannula was successfully introduced under multimodal image guidance. High resolution CBCT enabled validation of the cannula position and Gd-enhanced MRI after CED confirmed localized administration of the therapy. Conclusion A microcannula for CED was introduced into the striatum of five sheep under multimodal image guidance. The non-alloy 300 μm diameter cannula tip was well visualized using CBCT, enabling confirmation of the position of the end of the tip in the area of interest. PMID:22193239

GRAFT-VERSUS-HOST DISEASE PANUVEITIS AND BILATERAL SEROUS DETACHMENTS: MULTIMODAL IMAGING ANALYSIS.

PubMed

Jung, Jesse J; Chen, Michael H; Rofagha, Soraya; Lee, Scott S

2017-01-01

To report the multimodal imaging findings and follow-up of a case of graft-versus-host disease-induced bilateral panuveitis and serous retinal detachments after allogenic bone marrow transplant for acute myeloid leukemia. A 75-year-old black man presented with acute decreased vision in both eyes for 1 week. Clinical examination and multimodal imaging, including spectral domain optical coherence tomography, fundus autofluorescence, fluorescein angiography, and swept-source optical coherence tomography angiography (Investigational Device; Carl Zeiss Meditec Inc) were performed. Clinical examination of the patient revealed anterior and posterior inflammation and bilateral serous retinal detachments. Ultra-widefield fundus autofluorescence demonstrated hyperautofluorescence secondary to subretinal fluid; and fluorescein angiography revealed multiple areas of punctate hyperfluorescence, leakage, and staining of the optic discs. Spectral domain and enhanced depth imaging optical coherence tomography demonstrated subretinal fluid, a thickened, undulating retinal pigment epithelium layer, and a thickened choroid in both eyes. En-face swept-source optical coherence tomography angiography did not show any retinal vascular abnormalities but did demonstrate patchy areas of decreased choriocapillaris flow. An extensive systemic infectious and malignancy workup was negative and the patient was treated with high-dose oral prednisone immunosuppression. Subsequent 6-month follow-up demonstrated complete resolution of the inflammation and bilateral serous detachments after completion of the prednisone taper over a 3-month period. Graft-versus-host disease panuveitis and bilateral serous retinal detachments are rare complications of allogenic bone marrow transplant for acute myeloid leukemia and can be diagnosed with clinical and multimodal imaging analysis. This form of autoimmune inflammation may occur after the recovery of T-cell activity within the donor graft targeting the host. Infectious and recurrent malignancy must be ruled out before initiation of immunosuppression, which can affectively treat this form of graft-versus-host disease.

Morphology supporting function: attenuation correction for SPECT/CT, PET/CT, and PET/MR imaging

PubMed Central

Lee, Tzu C.; Alessio, Adam M.; Miyaoka, Robert M.; Kinahan, Paul E.

2017-01-01

Both SPECT, and in particular PET, are unique in medical imaging for their high sensitivity and direct link to a physical quantity, i.e. radiotracer concentration. This gives PET and SPECT imaging unique capabilities for accurately monitoring disease activity for the purposes of clinical management or therapy development. However, to achieve a direct quantitative connection between the underlying radiotracer concentration and the reconstructed image values several confounding physical effects have to be estimated, notably photon attenuation and scatter. With the advent of dual-modality SPECT/CT, PET/CT, and PET/MR scanners, the complementary CT or MR image data can enable these corrections, although there are unique challenges for each combination. This review covers the basic physics underlying photon attenuation and scatter and summarizes technical considerations for multimodal imaging with regard to PET and SPECT quantification and methods to address the challenges for each multimodal combination. PMID:26576737

CHARACTERIZING PHOTORECEPTOR CHANGES IN ACUTE POSTERIOR MULTIFOCAL PLACOID PIGMENT EPITHELIOPATHY USING ADAPTIVE OPTICS.

PubMed

Roberts, Philipp K; Nesper, Peter L; Onishi, Alex C; Skondra, Dimitra; Jampol, Lee M; Fawzi, Amani A

2018-01-01

To characterize lesions of acute posterior multifocal placoid pigment epitheliopathy (APMPPE) by multimodal imaging including adaptive optics scanning laser ophthalmoscopy (AOSLO). We included patients with APMPPE at different stages of evolution of the placoid lesions. Color fundus photography, spectral domain optical coherence tomography, infrared reflectance, fundus autofluorescence, and AOSLO images were obtained and registered to correlate microstructural changes. Eight eyes of four patients (two women) were included and analyzed by multimodal imaging. Photoreceptor reflectivity within APMPPE lesions was more heterogeneous than in adjacent healthy areas. Hyperpigmentation on color fundus photography appeared hyperreflective on infrared reflectance and on AOSLO. Irregularity of the interdigitation zone and the photoreceptor inner and outer segment junctions (IS/OS) on spectral domain optical coherence tomography was associated with photoreceptor hyporeflectivity on AOSLO. Interruption of the interdigitation zone or IS/OS was associated with loss of photoreceptor reflectivity on AOSLO. Irregularities in the reflectivity of the photoreceptor mosaic are visible on AOSLO even in inactive APMPPE lesions, where the photoreceptor bands on spectral domain optical coherence tomography have recovered. Adaptive optics scanning laser ophthalmoscopy combined with multimodal imaging has the potential to enhance our understanding of photoreceptor involvement in APMPPE.

Tissue imaging using full field optical coherence microscopy with short multimode fiber probe

NASA Astrophysics Data System (ADS)

Sato, Manabu; Eto, Kai; Goto, Tetsuhiro; Kurotani, Reiko; Abe, Hiroyuki; Nishidate, Izumi

2018-03-01

In achieving minimally invasive accessibility to deeply located regions the size of the imaging probes is important. We demonstrated full-field optical coherence tomography (FF-OCM) using an ultrathin forward-imaging short multimode fiber (SMMF) probe of 50 μm core diameter, 125 μm diameter, and 7.4 mm length for optical communications. The axial resolution was measured to be 2.14 μm and the lateral resolution was also evaluated to be below 4.38 μm using a test pattern (TP). The spatial mode and polarization characteristics of SMMF were evaluated. Inserting SMMF to in vivo rat brain, 3D images were measured and 2D information of nerve fibers was obtained. The feasibility of an SMMF as an ultrathin forward-imaging probe in FF-OCM has been demonstrated.

vECTlab—A fully integrated multi-modality Monte Carlo simulation framework for the radiological imaging sciences

NASA Astrophysics Data System (ADS)

Peter, Jörg; Semmler, Wolfhard

2007-10-01

Alongside and in part motivated by recent advances in molecular diagnostics, the development of dual-modality instruments for patient and dedicated small animal imaging has gained attention by diverse research groups. The desire for such systems is high not only to link molecular or functional information with the anatomical structures, but also for detecting multiple molecular events simultaneously at shorter total acquisition times. While PET and SPECT have been integrated successfully with X-ray CT, the advance of optical imaging approaches (OT) and the integration thereof into existing modalities carry a high application potential, particularly for imaging small animals. A multi-modality Monte Carlo (MC) simulation approach at present has been developed that is able to trace high-energy (keV) as well as optical (eV) photons concurrently within identical phantom representation models. We show that the involved two approaches for ray-tracing keV and eV photons can be integrated into a unique simulation framework which enables both photon classes to be propagated through various geometry models representing both phantoms and scanners. The main advantage of such integrated framework for our specific application is the investigation of novel tomographic multi-modality instrumentation intended for in vivo small animal imaging through time-resolved MC simulation upon identical phantom geometries. Design examples are provided for recently proposed SPECT-OT and PET-OT imaging systems.

"Where Lies Your Text?" ("Twelfth Night" Act I, Scene V): Engaging High School Students from Low Socioeconomic Backgrounds in Reading Multimodal Texts

ERIC Educational Resources Information Center

Callow, Jon; Zammit, Katina

2012-01-01

From the original performances of Shakespeare's plays and the illustrated title page of his First Folios, to the current day use of YouTube, mobile phones and tablets, multimodal texts have been apparent across Western cultures. The rapid development over the last 20 years of technology has markedly increased access to a much broader array of…

Multimodal autofluorescence detection of cancer: from single cells to living organism

NASA Astrophysics Data System (ADS)

Horilova, J.; Cunderlikova, B.; Cagalinec, M.; Chorvat, D.; Marcek Chorvatova, A.

2018-02-01

Multimodal optical imaging of suspected tissues is showing to be a promising method for distinguishing suspected cancerous tissues from healthy ones. In particular, the combination of steady-state spectroscopic methods with timeresolved fluorescence provides more precise insight into native metabolism when focused on tissue autofluorescence. Cancer is linked to specific metabolic remodelation detectable spectroscopically. In this work, we evaluate possibilities and limitations of multimodal optical cancer detection in single cells, collagen-based 3D cell cultures and in living organisms (whole mice), as a representation of gradually increasing complexity of model systems.

Generation of a suite of 3D computer-generated breast phantoms from a limited set of human subject data

DOE Office of Scientific and Technical Information (OSTI.GOV)

Hsu, Christina M. L.; Palmeri, Mark L.; Department of Anesthesiology, Duke University Medical Center, Durham, North Carolina 27710

2013-04-15

Purpose: The authors previously reported on a three-dimensional computer-generated breast phantom, based on empirical human image data, including a realistic finite-element based compression model that was capable of simulating multimodality imaging data. The computerized breast phantoms are a hybrid of two phantom generation techniques, combining empirical breast CT (bCT) data with flexible computer graphics techniques. However, to date, these phantoms have been based on single human subjects. In this paper, the authors report on a new method to generate multiple phantoms, simulating additional subjects from the limited set of original dedicated breast CT data. The authors developed an image morphingmore » technique to construct new phantoms by gradually transitioning between two human subject datasets, with the potential to generate hundreds of additional pseudoindependent phantoms from the limited bCT cases. The authors conducted a preliminary subjective assessment with a limited number of observers (n= 4) to illustrate how realistic the simulated images generated with the pseudoindependent phantoms appeared. Methods: Several mesh-based geometric transformations were developed to generate distorted breast datasets from the original human subject data. Segmented bCT data from two different human subjects were used as the 'base' and 'target' for morphing. Several combinations of transformations were applied to morph between the 'base' and 'target' datasets such as changing the breast shape, rotating the glandular data, and changing the distribution of the glandular tissue. Following the morphing, regions of skin and fat were assigned to the morphed dataset in order to appropriately assign mechanical properties during the compression simulation. The resulting morphed breast was compressed using a finite element algorithm and simulated mammograms were generated using techniques described previously. Sixty-two simulated mammograms, generated from morphing three human subject datasets, were used in a preliminary observer evaluation where four board certified breast radiologists with varying amounts of experience ranked the level of realism (from 1 ='fake' to 10 ='real') of the simulated images. Results: The morphing technique was able to successfully generate new and unique morphed datasets from the original human subject data. The radiologists evaluated the realism of simulated mammograms generated from the morphed and unmorphed human subject datasets and scored the realism with an average ranking of 5.87 {+-} 1.99, confirming that overall the phantom image datasets appeared more 'real' than 'fake.' Moreover, there was not a significant difference (p > 0.1) between the realism of the unmorphed datasets (6.0 {+-} 1.95) compared to the morphed datasets (5.86 {+-} 1.99). Three of the four observers had overall average rankings of 6.89 {+-} 0.89, 6.9 {+-} 1.24, 6.76 {+-} 1.22, whereas the fourth observer ranked them noticeably lower at 2.94 {+-} 0.7. Conclusions: This work presents a technique that can be used to generate a suite of realistic computerized breast phantoms from a limited number of human subjects. This suite of flexible breast phantoms can be used for multimodality imaging research to provide a known truth while concurrently producing realistic simulated imaging data.« less

MULTIPLE EVANESCENT WHITE DOT SYNDROME WITH SUBRETINAL DEPOSITS.

PubMed

Gal-Or, Orly; Sorenson, John A; Gattoussi, Sarra; Dolz-Marco, Rosa; Freund, K Bailey

2017-06-13

To describe the multimodal imaging findings of transient subretinal deposits occurring in multiple evanescent white dot syndrome (MEWDS). The multimodal imaging characteristics of transient subretinal deposits occurring in MEWDS were investigated with ultra-widefield color and fundus autofluorescence, cross-sectional and en-face optical coherence tomography (OCT), en face OCT-angiography, and quantitative autofluorescence. A 28-year-old woman presented with photopsia and temporal visual field loss in her right eye. Her best-corrected visual acuity was 20/20 in her right eye and 20/25 in her left eye. Funduscopic examination showed characteristic peripapillary hyperautofluorescent white dots of MEWDS corresponding to ellipsoid zone disruption on OCT. These lesions became confluent throughout the posterior fundus over the next 4 weeks. As the patient's symptoms were resolving, a second type of transient hyperautofluorescent lesion was noted which corresponded to hyperreflective subretinal deposits on cross-sectional and en face structural OCT. These subretinal deposits were most evident at 10-week follow-up and had nearly resolved at 14-week follow-up. Quantitative autofluorescence showed that, unlike the acute MEWDS lesions, the hyperautoflurescence of the subretinal deposits persisted after photobleaching. At multiple time points over 14 weeks of follow-up, OCT angiography showed no evidence of retinal or choroidal flow abnormalities. Transient subretinal deposits may develop during MEWDS in areas of previous diffuse outer retinal disruption. As these deposits remain hyperautoflurescent on quantitative autofluorescence after photobleaching, they may represent accumulations of debris originating from damaged photoreceptor outer segments.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kamran, Mudassar, E-mail: kamranm@mir.wustl.edu; Fowler, Kathryn J., E-mail: fowlerk@mir.wustl.edu; Mellnick, Vincent M., E-mail: mellnickv@mir.wustl.edu

Primary aortic neoplasms are rare. Aortic sarcoma arising after endovascular aneurysm repair (EVAR) is a scarce subset of primary aortic malignancies, reports of which are infrequent in the published literature. The diagnosis of aortic sarcoma is challenging due to its non-specific clinical presentation, and the prognosis is poor due to delayed diagnosis, rapid proliferation, and propensity for metastasis. Post-EVAR, aortic sarcomas may mimic other more common aortic processes on surveillance imaging. Radiologists are rarely knowledgeable about this rare entity for which multimodality imaging and awareness are invaluable in early diagnosis. A series of three pathologically confirmed cases are presented tomore » display the multimodality imaging features and clinical presentations of aortic sarcoma arising after EVAR.« less

Multimodal device for assessment of skin malformations

NASA Astrophysics Data System (ADS)

Bekina, A.; Garancis, V.; Rubins, U.; Spigulis, J.; Valeine, L.; Berzina, A.

2013-11-01

A variety of multi-spectral imaging devices is commercially available and used for skin diagnostics and monitoring; however, an alternative cost-efficient device can provide an advanced spectral analysis of skin. A compact multimodal device for diagnosis of pigmented skin lesions was developed and tested. A polarized LED light source illuminates the skin surface at four different wavelengths - blue (450 nm), green (545 nm), red (660 nm) and infrared (940 nm). Spectra of reflected light from the 25 mm wide skin spot are imaged by a CMOS sensor. Four spectral images are obtained for mapping of the main skin chromophores. The specific chromophore distribution differences between different skin malformations were analyzed and information of subcutaneous structures was consecutively extracted.

Multimodal instrument for high-sensitivity autofluorescence and spectral optical coherence tomography of the human eye fundus

PubMed Central

Komar, Katarzyna; Stremplewski, Patrycjusz; Motoczyńska, Marta; Szkulmowski, Maciej; Wojtkowski, Maciej

2013-01-01

In this paper we present a multimodal device for imaging fundus of human eye in vivo which combines functionality of autofluorescence by confocal SLO with Fourier domain OCT. Native fluorescence of human fundus was excited by modulated laser beam (λ = 473 nm, 20 MHz) and lock-in detection was applied resulting in improving sensitivity. The setup allows for acquisition of high resolution OCT and high contrast AF images using fluorescence excitation power of 50-65 μW without averaging consecutive images. Successful functioning of constructed device have been demonstrated for 8 healthy volunteers of different age ranging from 24 to 83 years old. PMID:24298426

Multifunctional fluorescent and magnetic nanoparticles for biomedical applications

NASA Astrophysics Data System (ADS)

Selvan, Subramanian T.

2012-03-01

Hybrid multifunctional nanoparticles (NPs) are emerging as useful probes for magnetic based targeting, delivery, cell separation, magnetic resonance imaging (MRI), and fluorescence-based bio-labeling applications. Assessing from the literature, the development of multifunctional NPs for multimodality imaging is still in its infancy state. This report focuses on our recent work on quantum dots (QDs), magnetic NPs (MNPs) and bi-functional NPs (composed of either QDs or rare-earth NPs, and magnetic NPs - iron oxide or gadolinium oxide) for multimodality imaging based biomedical applications. The combination of MRI and fluorescence would ally each other in improving the sensitivity and resolution, resulting in improved and early diagnosis of the disease. The challenges in this area are discussed.

Resolution and throughput optimized intraoperative spectrally encoded coherence tomography and reflectometry (iSECTR) for multimodal imaging during ophthalmic microsurgery

NASA Astrophysics Data System (ADS)

Malone, Joseph D.; El-Haddad, Mohamed T.; Leeburg, Kelsey C.; Terrones, Benjamin D.; Tao, Yuankai K.

2018-02-01

Limited visualization of semi-transparent structures in the eye remains a critical barrier to improving clinical outcomes and developing novel surgical techniques. While increases in imaging speed has enabled intraoperative optical coherence tomography (iOCT) imaging of surgical dynamics, several critical barriers to clinical adoption remain. Specifically, these include (1) static field-of-views (FOVs) requiring manual instrument-tracking; (2) high frame-rates require sparse sampling, which limits FOV; and (3) small iOCT FOV also limits the ability to co-register data with surgical microscopy. We previously addressed these limitations in image-guided ophthalmic microsurgery by developing microscope-integrated multimodal intraoperative swept-source spectrally encoded scanning laser ophthalmoscopy and optical coherence tomography. Complementary en face images enabled orientation and coregistration with the widefield surgical microscope view while OCT imaging enabled depth-resolved visualization of surgical instrument positions relative to anatomic structures-of-interest. In addition, we demonstrated novel integrated segmentation overlays for augmented-reality surgical guidance. Unfortunately, our previous system lacked the resolution and optical throughput for in vivo retinal imaging and necessitated removal of cornea and lens. These limitations were predominately a result of optical aberrations from imaging through a shared surgical microscope objective lens, which was modeled as a paraxial surface. Here, we present an optimized intraoperative spectrally encoded coherence tomography and reflectometry (iSECTR) system. We use a novel lens characterization method to develop an accurate model of surgical microscope objective performance and balance out inherent aberrations using iSECTR relay optics. Using this system, we demonstrate in vivo multimodal ophthalmic imaging through a surgical microscope

Multimodality image integration for radiotherapy treatment: an easy approach

NASA Astrophysics Data System (ADS)

Santos, Andres; Pascau, Javier; Desco, Manuel; Santos, Juan A.; Calvo, Felipe A.; Benito, Carlos; Garcia-Barreno, Rafael

2001-05-01

The interest of using combined MR and CT information for radiotherapy planning is well documented. However, many planning workstations do not allow to use MR images, nor import predefined contours. This paper presents a new simple approach for transferring segmentation results from MRI to a CT image that will be used for radiotherapy planning, using the same original CT format. CT and MRI images of the same anatomical area are registered using mutual information (MI) algorithm. Targets and organs at risk are segmented by the physician on the MR image, where their contours are easy to track. A locally developed software running on PC is used for this step, with several facilities for the segmentation process. The result is transferred onto the CT by slightly modifying up and down the original Hounsfield values of some points of the contour. This is enough to visualize the contour on the CT, but does not affect dose calculations. The CT is then stored using the original file format of the radiotherapy planning workstation, where the technician uses the segmented contour to design the correct beam positioning. The described method has been tested in five patients. Simulations and patient results show that the dose distribution is not affected by the small modification of pixels of the CT image, while the segmented structures can be tracked in the radiotherapy planning workstation-using adequate window/level settings. The presence of the physician is not requires at the planning workstation, and he/she can perform the segmentation process using his/her own PC. This new approach makes it possible to take advantage from the anatomical information present on the MRI and to transfer the segmentation to the CT used for planning, even when the planning workstation does not allow to import external contours. The physician can draw the limits of the target and areas at risk off-line, thus separating in time the segmentation and planning tasks and increasing the efficiency.

Molecular Imaging and Therapy of Prostate Cancer

DTIC Science & Technology

2015-10-01

arsenic-based, IGF1R-targeted radiopharmaceuticals can allow for PET imaging, IRT, and monitoring the therapeutic response of PCa. Specific Aims: Aim 1: To...models with PET imaging. Aim 3: To monitor the efficacy of 76As-based IRT of PCa with multimodality imaging.

Use of multidimensional, multimodal imaging and PACS to support neurological diagnoses

NASA Astrophysics Data System (ADS)

Wong, Stephen T. C.; Knowlton, Robert C.; Hoo, Kent S.; Huang, H. K.

1995-05-01

Technological advances in brain imaging have revolutionized diagnosis in neurology and neurological surgery. Major imaging techniques include magnetic resonance imaging (MRI) to visualize structural anatomy, positron emission tomography (PET) to image metabolic function and cerebral blood flow, magnetoencephalography (MEG) to visualize the location of physiologic current sources, and magnetic resonance spectroscopy (MRS) to measure specific biochemicals. Each of these techniques studies different biomedical aspects of the brain, but there lacks an effective means to quantify and correlate the disparate imaging datasets in order to improve clinical decision making processes. This paper describes several techniques developed in a UNIX-based neurodiagnostic workstation to aid the noninvasive presurgical evaluation of epilepsy patients. These techniques include online access to the picture archiving and communication systems (PACS) multimedia archive, coregistration of multimodality image datasets, and correlation and quantitation of structural and functional information contained in the registered images. For illustration, we describe the use of these techniques in a patient case of nonlesional neocortical epilepsy. We also present out future work based on preliminary studies.

Multimodal Medical Image Fusion by Adaptive Manifold Filter.

PubMed

Geng, Peng; Liu, Shuaiqi; Zhuang, Shanna

2015-01-01

Medical image fusion plays an important role in diagnosis and treatment of diseases such as image-guided radiotherapy and surgery. The modified local contrast information is proposed to fuse multimodal medical images. Firstly, the adaptive manifold filter is introduced into filtering source images as the low-frequency part in the modified local contrast. Secondly, the modified spatial frequency of the source images is adopted as the high-frequency part in the modified local contrast. Finally, the pixel with larger modified local contrast is selected into the fused image. The presented scheme outperforms the guided filter method in spatial domain, the dual-tree complex wavelet transform-based method, nonsubsampled contourlet transform-based method, and four classic fusion methods in terms of visual quality. Furthermore, the mutual information values by the presented method are averagely 55%, 41%, and 62% higher than the three methods and those values of edge based similarity measure by the presented method are averagely 13%, 33%, and 14% higher than the three methods for the six pairs of source images.

Ions doped melanin nanoparticle as a multiple imaging agent.

PubMed

Ha, Shin-Woo; Cho, Hee-Sang; Yoon, Young Il; Jang, Moon-Sun; Hong, Kwan Soo; Hui, Emmanuel; Lee, Jung Hee; Yoon, Tae-Jong

2017-10-10

Multimodal nanomaterials are useful for providing enhanced diagnostic information simultaneously for a variety of in vivo imaging methods. According to our research findings, these multimodal nanomaterials offer promising applications for cancer therapy. Melanin nanoparticles can be used as a platform imaging material and they can be simply produced by complexation with various imaging active ions. They are capable of specifically targeting epidermal growth factor receptor (EGFR)-expressing cancer cells by being anchored with a specific antibody. Ion-doped melanin nanoparticles were found to have high bioavailability with long-term stability in solution, without any cytotoxicity in both in vitro and in vivo systems. By combining different imaging modalities with melanin particles, we can use the complexes to obtain faster diagnoses by computed tomography deep-body imaging and greater detailed pathological diagnostic information by magnetic resonance imaging. The ion-doped melanin nanoparticles also have applications for radio-diagnostic treatment and radio imaging-guided surgery, warranting further proof of concept experimental.

Multimodal imaging system for dental caries detection

NASA Astrophysics Data System (ADS)

Liang, Rongguang; Wong, Victor; Marcus, Michael; Burns, Peter; McLaughlin, Paul

2007-02-01

Dental caries is a disease in which minerals of the tooth are dissolved by surrounding bacterial plaques. A caries process present for some time may result in a caries lesion. However, if it is detected early enough, the dentist and dental professionals can implement measures to reverse and control caries. Several optical, nonionized methods have been investigated and used to detect dental caries in early stages. However, there is not a method that can singly detect the caries process with both high sensitivity and high specificity. In this paper, we present a multimodal imaging system that combines visible reflectance, fluorescence, and Optical Coherence Tomography (OCT) imaging. This imaging system is designed to obtain one or more two-dimensional images of the tooth (reflectance and fluorescence images) and a three-dimensional OCT image providing depth and size information of the caries. The combination of two- and three-dimensional images of the tooth has the potential for highly sensitive and specific detection of dental caries.

Multimodal Imaging of Brain Connectivity Using the MIBCA Toolbox: Preliminary Application to Alzheimer's Disease

NASA Astrophysics Data System (ADS)

Ribeiro, André Santos; Lacerda, Luís Miguel; Silva, Nuno André da; Ferreira, Hugo Alexandre

2015-06-01

The Multimodal Imaging Brain Connectivity Analysis (MIBCA) toolbox is a fully automated all-in-one connectivity analysis toolbox that offers both pre-processing, connectivity, and graph theory analysis of multimodal images such as anatomical, diffusion, and functional MRI, and PET. In this work, the MIBCA functionalities were used to study Alzheimer's Disease (AD) in a multimodal MR/PET approach. Materials and Methods: Data from 12 healthy controls, and 36 patients with EMCI, LMCI and AD (12 patients for each group) were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu), including T1-weighted (T1-w), Diffusion Tensor Imaging (DTI) data, and 18F-AV-45 (florbetapir) dynamic PET data from 40-60 min post injection (4x5 min). Both MR and PET data were automatically pre-processed for all subjects using MIBCA. T1-w data was parcellated into cortical and subcortical regions-of-interest (ROIs), and the corresponding thicknesses and volumes were calculated. DTI data was used to compute structural connectivity matrices based on fibers connecting pairs of ROIs. Lastly, dynamic PET images were summed, and the relative Standard Uptake Values calculated for each ROI. Results: An overall higher uptake of 18F-AV-45, consistent with an increased deposition of beta-amyloid, was observed for the AD group. Additionally, patients showed significant cortical atrophy (thickness and volume) especially in the entorhinal cortex and temporal areas, and a significant increase in Mean Diffusivity (MD) in the hippocampus, amygdala and temporal areas. Furthermore, patients showed a reduction of fiber connectivity with the progression of the disease, especially for intra-hemispherical connections. Conclusion: This work shows the potential of the MIBCA toolbox for the study of AD, as findings were shown to be in agreement with the literature. Here, only structural changes and beta-amyloid accumulation were considered. Yet, MIBCA is further able to process fMRI and different radiotracers, thus leading to integration of functional information, and supporting the research for new multimodal biomarkers for AD and other neurodegenerative diseases.

Evaluation of the role of CD207 on Langerhans cells in a murine model of atopic dermatitis by in situ imaging using Cr:forsterite laser-based multimodality nonlinear microscopy

NASA Astrophysics Data System (ADS)

Lee, Jyh-Hong; Tsai, Ming-Rung; Sun, Chi-Kuang; Chiang, Bor-Luen

2012-11-01

Atopic dermatitis (AD) is an allergic inflammatory disease of skin. It remains unclear that CD207 of Langerhans cells (LCs) plays a central role in the development of allergic sensitization. There is little data on LCs within the microenviroment in vivo. We used a murine model of epicutaneous (EC) ovalbumin (OVA) sensitization inducing an inflammatory skin resembling AD to explore the role of CD207 in the pathogenesis of AD. Cr:forsterite laser-based multimodality nonlinear microscopy was applied for in situ imaging. Peritoneal injections of Alexa Fluor 647-rat anti-mouse CD207 into mice were performed to specifically trace the LCs. Peritoneal injections of OVA-Alexa Fluor 647 conjugate into mice were performed to specifically trace the OVA. We found that combining Alexa Fluor fluorescent probes with multimodality nonlinear microscopy permitted the unequivocal in situ imaging of CD207-expressing LCs. The relevant time-course, expressional, and functional studies reveal that CD207 of LCs plays an essential role during the induction of EC sensitization. We establish and validate that Cr:forsterite laser-based multimodality nonlinear microscopy is applicable for the specific detection of labeled mAb-bound LCs and labeled antigen. We suggest that CD207-expressing LCs initiate the allergic response through the CD207 mediated epicutaneous sensitization associated with the development of AD.

PET-CMR in heart failure - synergistic or redundant imaging?

PubMed

Quail, Michael A; Sinusas, Albert J

2017-07-01

Imaging in heart failure (HF) provides data for diagnosis, prognosis and disease monitoring. Both MRI and nuclear imaging techniques have been successfully used for this purpose in HF. Positron Emission Tomography-Cardiac Magnetic Resonance (PET-CMR) is an example of a new multimodality diagnostic imaging technique with potential applications in HF. The threshold for adopting a new diagnostic tool to clinical practice must necessarily be high, lest they exacerbate costs without improving care. New modalities must demonstrate clinical superiority, or at least equivalence, combined with another important advantage, such as lower cost or improved patient safety. The purpose of this review is to outline the current status of multimodality PET-CMR with regard to HF applications, and determine whether the clinical utility of this new technology justifies the cost.

Multimodal nonlinear imaging of arabidopsis thaliana root cell

NASA Astrophysics Data System (ADS)

Jang, Bumjoon; Lee, Sung-Ho; Woo, Sooah; Park, Jong-Hyun; Lee, Myeong Min; Park, Seung-Han

2017-07-01

Nonlinear optical microscopy has enabled the possibility to explore inside the living organisms. It utilizes ultrashort laser pulse with long wavelength (greater than 800nm). Ultrashort pulse produces high peak power to induce nonlinear optical phenomenon such as two-photon excitation fluorescence (TPEF) and harmonic generations in the medium while maintaining relatively low average energy pre area. In plant developmental biology, confocal microscopy is widely used in plant cell imaging after the development of biological fluorescence labels in mid-1990s. However, fluorescence labeling itself affects the sample and the sample deviates from intact condition especially when labelling the entire cell. In this work, we report the dynamic images of Arabidopsis thaliana root cells. This demonstrates the multimodal nonlinear optical microscopy is an effective tool for long-term plant cell imaging.

Simultaneous acquisition of magnetic resonance spectroscopy (MRS) data and positron emission tomography (PET) images with a prototype MR-compatible, small animal PET imager

NASA Astrophysics Data System (ADS)

Raylman, Raymond R.; Majewski, Stan; Velan, S. Sendhil; Lemieux, Susan; Kross, Brian; Popov, Vladimir; Smith, Mark F.; Weisenberger, Andrew G.

2007-06-01

Multi-modality imaging (such as PET-CT) is rapidly becoming a valuable tool in the diagnosis of disease and in the development of new drugs. Functional images produced with PET, fused with anatomical images created by MRI, allow the correlation of form with function. Perhaps more exciting than the combination of anatomical MRI with PET, is the melding of PET with MR spectroscopy (MRS). Thus, two aspects of physiology could be combined in novel ways to produce new insights into the physiology of normal and pathological processes. Our team is developing a system to acquire MRI images and MRS spectra, and PET images contemporaneously. The prototype MR-compatible PET system consists of two opposed detector heads (appropriate in size for small animal imaging), operating in coincidence mode with an active field-of-view of ˜14 cm in diameter. Each detector consists of an array of LSO detector elements coupled through a 2-m long fiber optic light guide to a single position-sensitive photomultiplier tube. The use of light guides allows these magnetic field-sensitive elements of the PET imager to be positioned outside the strong magnetic field of our 3T MRI scanner. The PET scanner imager was integrated with a 12-cm diameter, 12-leg custom, birdcage coil. Simultaneous MRS spectra and PET images were successfully acquired from a multi-modality phantom consisting of a sphere filled with 17 brain relevant substances and a positron-emitting radionuclide. There were no significant changes in MRI or PET scanner performance when both were present in the MRI magnet bore. This successful initial test demonstrates the potential for using such a multi-modality to obtain complementary MRS and PET data.

Multimode C-arm fluoroscopy, tomosynthesis, and cone-beam CT for image-guided interventions: from proof of principle to patient protocols

NASA Astrophysics Data System (ADS)

Siewerdsen, J. H.; Daly, M. J.; Bachar, G.; Moseley, D. J.; Bootsma, G.; Brock, K. K.; Ansell, S.; Wilson, G. A.; Chhabra, S.; Jaffray, D. A.; Irish, J. C.

2007-03-01

High-performance intraoperative imaging is essential to an ever-expanding scope of therapeutic procedures ranging from tumor surgery to interventional radiology. The need for precise visualization of bony and soft-tissue structures with minimal obstruction to the therapy setup presents challenges and opportunities in the development of novel imaging technologies specifically for image-guided procedures. Over the past ~5 years, a mobile C-arm has been modified in collaboration with Siemens Medical Solutions for 3D imaging. Based upon a Siemens PowerMobil, the device includes: a flat-panel detector (Varian PaxScan 4030CB); a motorized orbit; a system for geometric calibration; integration with real-time tracking and navigation (NDI Polaris); and a computer control system for multi-mode fluoroscopy, tomosynthesis, and cone-beam CT. Investigation of 3D imaging performance (noise-equivalent quanta), image quality (human observer studies), and image artifacts (scatter, truncation, and cone-beam artifacts) has driven the development of imaging techniques appropriate to a host of image-guided interventions. Multi-mode functionality presents a valuable spectrum of acquisition techniques: i.) fluoroscopy for real-time 2D guidance; ii.) limited-angle tomosynthesis for fast 3D imaging (e.g., ~10 sec acquisition of coronal slices containing the surgical target); and iii.) fully 3D cone-beam CT (e.g., ~30-60 sec acquisition providing bony and soft-tissue visualization across the field of view). Phantom and cadaver studies clearly indicate the potential for improved surgical performance - up to a factor of 2 increase in challenging surgical target excisions. The C-arm system is currently being deployed in patient protocols ranging from brachytherapy to chest, breast, spine, and head and neck surgery.

Adaptive wavefront shaping for controlling nonlinear multimode interactions in optical fibres

NASA Astrophysics Data System (ADS)

Tzang, Omer; Caravaca-Aguirre, Antonio M.; Wagner, Kelvin; Piestun, Rafael

2018-06-01

Recent progress in wavefront shaping has enabled control of light propagation inside linear media to focus and image through scattering objects. In particular, light propagation in multimode fibres comprises complex intermodal interactions and rich spatiotemporal dynamics. Control of physical phenomena in multimode fibres and its applications are in their infancy, opening opportunities to take advantage of complex nonlinear modal dynamics. Here, we demonstrate a wavefront shaping approach for controlling nonlinear phenomena in multimode fibres. Using a spatial light modulator at the fibre input, real-time spectral feedback and a genetic algorithm optimization, we control a highly nonlinear multimode stimulated Raman scattering cascade and its interplay with four-wave mixing via a flexible implicit control on the superposition of modes coupled into the fibre. We show versatile spectrum manipulations including shifts, suppression, and enhancement of Stokes and anti-Stokes peaks. These demonstrations illustrate the power of wavefront shaping to control and optimize nonlinear wave propagation.

A Remaking Pedagogy: Adaptation and Archetypes in the Child's Multimodal Reading and Writing

ERIC Educational Resources Information Center

Berger, Richard; Zezulkova, Marketa

2018-01-01

This paper proposes combining theories about, and practices of, using archetypes and adaptation in education for the purposes of multimodal literacy learning. Within such contexts, children of primary school age act as readers, performers and researchers, exploring and analysing existing adaptations of archetypal stories and images across time,…

Multimodal MRI for early diabetic mild cognitive impairment: study protocol of a prospective diagnostic trial.

PubMed

Yu, Ying; Sun, Qian; Yan, Lin-Feng; Hu, Yu-Chuan; Nan, Hai-Yan; Yang, Yang; Liu, Zhi-Cheng; Wang, Wen; Cui, Guang-Bin

2016-08-24

Type 2 diabetes mellitus (T2DM) is a risk factor for dementia. Mild cognitive impairment (MCI), an intermediary state between normal cognition and dementia, often occurs during the prodromal diabetic stage, making early diagnosis and intervention of MCI very important. Latest neuroimaging techniques revealed some underlying microstructure alterations for diabetic MCI, from certain aspects. But there still lacks an integrated multimodal MRI system to detect early neuroimaging changes in diabetic MCI patients. Thus, we intended to conduct a diagnostic trial using multimodal MRI techniques to detect early diabetic MCI that is determined by the Montreal Cognitive Assessment (MoCA). In this study, healthy controls, prodromal diabetes and diabetes subjects (53 subjects/group) aged 40-60 years will be recruited from the physical examination center of Tangdu Hospital. The neuroimaging and psychometric measurements will be repeated at a 0.5 year-interval for 2.5 years' follow-up. The primary outcome measures are 1) Microstructural and functional alterations revealed with multimodal MRI scans including structure magnetic resonance imaging (sMRI), resting state functional magnetic resonance imaging (rs-fMRI), diffusion kurtosis imaging (DKI), and three-dimensional pseudo-continuous arterial spin labeling (3D-pCASL); 2) Cognition evaluation with MoCA. The second outcome measures are obesity, metabolic characteristics, lifestyle and quality of life. The study will provide evidence for the potential use of multimodal MRI techniques with psychometric evaluation in diagnosing MCI at prodromal diabetic stage so as to help decision making in early intervention and improve the prognosis of T2DM. This study has been registered to ClinicalTrials.gov ( NCT02420470 ) on April 2, 2015 and published on July 29, 2015.

On-road anomaly detection by multimodal sensor analysis and multimedia processing

NASA Astrophysics Data System (ADS)

Orhan, Fatih; Eren, P. E.

2014-03-01

The use of smartphones in Intelligent Transportation Systems is gaining popularity, yet many challenges exist in developing functional applications. Due to the dynamic nature of transportation, vehicular social applications face complexities such as developing robust sensor management, performing signal and image processing tasks, and sharing information among users. This study utilizes a multimodal sensor analysis framework which enables the analysis of sensors in multimodal aspect. It also provides plugin-based analyzing interfaces to develop sensor and image processing based applications, and connects its users via a centralized application as well as to social networks to facilitate communication and socialization. With the usage of this framework, an on-road anomaly detector is being developed and tested. The detector utilizes the sensors of a mobile device and is able to identify anomalies such as hard brake, pothole crossing, and speed bump crossing. Upon such detection, the video portion containing the anomaly is automatically extracted in order to enable further image processing analysis. The detection results are shared on a central portal application for online traffic condition monitoring.

Non-responders to cardiac resynchronization therapy: Insights from multimodality imaging and electrocardiography. A brief review.

PubMed

Carità, Patrizia; Corrado, Egle; Pontone, Gianluca; Curnis, Antonio; Bontempi, Luca; Novo, Giuseppina; Guglielmo, Marco; Ciaramitaro, Gianfranco; Assennato, Pasquale; Novo, Salvatore; Coppola, Giuseppe

2016-12-15

Cardiac resynchronization therapy (CRT) is a successful strategy for heart failure (HF) patients. The pre-requisite for the response is the evidence of electrical dyssynchrony on the surface electrocardiogram usually as left bundle branch block (LBBB). Non-response to CRT is a significant problem in clinical practice. Patient selection, inadequate delivery and sub-optimal left ventricle lead position may be important causes. In an effort to improve CRT response multimodality imaging (especially echocardiography, computed tomography and cardiac magnetic resonance) could play a decisive role and extensive literature has been published on the matter. However, we are so far from routinary use in clinical practice. Electrocardiography (with respect to left ventricle capture and QRS narrowing) may represent a simple and low cost approach for early prediction of potential non-responder, with immediate practical implications. This brief review covers the current recommendations for CRT in HF patients with particular attention to the potential benefits of multimodality imaging and electrocardiography in improving response rate. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Application of a multicompartment dynamical model to multimodal optical imaging for investigating individual cerebrovascular properties

NASA Astrophysics Data System (ADS)

Desjardins, Michèle; Gagnon, Louis; Gauthier, Claudine; Hoge, Rick D.; Dehaes, Mathieu; Desjardins-Crépeau, Laurence; Bherer, Louis; Lesage, Frédéric

2009-02-01

Biophysical models of hemodynamics provide a tool for quantitative multimodal brain imaging by allowing a deeper understanding of the interplay between neural activity and blood oxygenation, volume and flow responses to stimuli. Multicompartment dynamical models that describe the dynamics and interactions of the vascular and metabolic components of evoked hemodynamic responses have been developed in the literature. In this work, multimodal data using near-infrared spectroscopy (NIRS) and diffuse correlation flowmetry (DCF) is used to estimate total baseline hemoglobin concentration (HBT0) in 7 adult subjects. A validation of the model estimate and investigation of the partial volume effect is done by comparing with time-resolved spectroscopy (TRS) measures of absolute HBT0. Simultaneous NIRS and DCF measurements during hypercapnia are then performed, but are found to be hardly reproducible. The results raise questions about the feasibility of an all-optical model-based estimation of individual vascular properties.

Grid-Enabled Quantitative Analysis of Breast Cancer

DTIC Science & Technology

2010-10-01

large-scale, multi-modality computerized image analysis . The central hypothesis of this research is that large-scale image analysis for breast cancer...research, we designed a pilot study utilizing large scale parallel Grid computing harnessing nationwide infrastructure for medical image analysis . Also

Imaging the pleura.

PubMed

Mortensen, Chloe; Bhatnagar, Rahul; Edey, Anthony J

2012-11-01

Pleural disease is now recognized as an important subspecialty of pulmonary medicine, with increasing provision being made for specialist services and procedures. In response, the field of pleural imaging has advanced in recent years, especially with regard to ultrasound. Salient multimodality imaging techniques are discussed.

Simulation of brain tumors in MR images for evaluation of segmentation efficacy.

PubMed

Prastawa, Marcel; Bullitt, Elizabeth; Gerig, Guido

2009-04-01

Obtaining validation data and comparison metrics for segmentation of magnetic resonance images (MRI) are difficult tasks due to the lack of reliable ground truth. This problem is even more evident for images presenting pathology, which can both alter tissue appearance through infiltration and cause geometric distortions. Systems for generating synthetic images with user-defined degradation by noise and intensity inhomogeneity offer the possibility for testing and comparison of segmentation methods. Such systems do not yet offer simulation of sufficiently realistic looking pathology. This paper presents a system that combines physical and statistical modeling to generate synthetic multi-modal 3D brain MRI with tumor and edema, along with the underlying anatomical ground truth, Main emphasis is placed on simulation of the major effects known for tumor MRI, such as contrast enhancement, local distortion of healthy tissue, infiltrating edema adjacent to tumors, destruction and deformation of fiber tracts, and multi-modal MRI contrast of healthy tissue and pathology. The new method synthesizes pathology in multi-modal MRI and diffusion tensor imaging (DTI) by simulating mass effect, warping and destruction of white matter fibers, and infiltration of brain tissues by tumor cells. We generate synthetic contrast enhanced MR images by simulating the accumulation of contrast agent within the brain. The appearance of the the brain tissue and tumor in MRI is simulated by synthesizing texture images from real MR images. The proposed method is able to generate synthetic ground truth and synthesized MR images with tumor and edema that exhibit comparable segmentation challenges to real tumor MRI. Such image data sets will find use in segmentation reliability studies, comparison and validation of different segmentation methods, training and teaching, or even in evaluating standards for tumor size like the RECIST criteria (response evaluation criteria in solid tumors).

The multi-modal Australian ScienceS Imaging and Visualization Environment (MASSIVE) high performance computing infrastructure: applications in neuroscience and neuroinformatics research

PubMed Central

Goscinski, Wojtek J.; McIntosh, Paul; Felzmann, Ulrich; Maksimenko, Anton; Hall, Christopher J.; Gureyev, Timur; Thompson, Darren; Janke, Andrew; Galloway, Graham; Killeen, Neil E. B.; Raniga, Parnesh; Kaluza, Owen; Ng, Amanda; Poudel, Govinda; Barnes, David G.; Nguyen, Toan; Bonnington, Paul; Egan, Gary F.

2014-01-01

The Multi-modal Australian ScienceS Imaging and Visualization Environment (MASSIVE) is a national imaging and visualization facility established by Monash University, the Australian Synchrotron, the Commonwealth Scientific Industrial Research Organization (CSIRO), and the Victorian Partnership for Advanced Computing (VPAC), with funding from the National Computational Infrastructure and the Victorian Government. The MASSIVE facility provides hardware, software, and expertise to drive research in the biomedical sciences, particularly advanced brain imaging research using synchrotron x-ray and infrared imaging, functional and structural magnetic resonance imaging (MRI), x-ray computer tomography (CT), electron microscopy and optical microscopy. The development of MASSIVE has been based on best practice in system integration methodologies, frameworks, and architectures. The facility has: (i) integrated multiple different neuroimaging analysis software components, (ii) enabled cross-platform and cross-modality integration of neuroinformatics tools, and (iii) brought together neuroimaging databases and analysis workflows. MASSIVE is now operational as a nationally distributed and integrated facility for neuroinfomatics and brain imaging research. PMID:24734019

Multimode nonlinear optical imaging of the dermis in ex vivo human skin based on the combination of multichannel mode and Lambda mode.

PubMed

Zhuo, Shuangmu; Chen, Jianxin; Luo, Tianshu; Zou, Dingsong

2006-08-21

A Multimode nonlinear optical imaging technique based on the combination of multichannel mode and Lambda mode is developed to investigate human dermis. Our findings show that this technique not only improves the image contrast of the structural proteins of extracellular matrix (ECM) but also provides an image-guided spectral analysis method to identify both cellular and ECM intrinsic components including collagen, elastin, NAD(P)H and flavin. By the combined use of multichannel mode and Lambda mode in tandem, the obtained in-depth two photon-excited fluorescence (TPEF) and second-harmonic generation (SHG) imaging and TPEF/SHG signals depth-dependence decay can offer a sensitive tool for obtaining quantitative tissue structural and biochemical information. These results suggest that the technique has the potential to provide more accurate information for determining tissue physiological and pathological states.

Red fluorescent zinc oxide nanoparticle: A novel platform for cancer targeting

DOE PAGES

Hong, Hao; Wang, Fei; Zhang, Yin; ...

2015-01-21

Multifunctional zinc oxide (ZnO) nanoparticles (NPs) with well-integrated multimodality imaging capacities have generated increasing research interest in the past decade. However, limited progress has been made in developing ZnO NP-based multimodality tumor-imaging agents. In this paper, we developed novel red fluorescent ZnO NPs and described the successful conjugation of 64Cu ( t 1/2 = 12.7 h) and TRC105, a chimeric monoclonal antibody against CD105, to these ZnO NPs via well-developed surface engineering procedures. The produced dual-modality ZnO NPs were readily applicable for positron emission tomography (PET) imaging and fluorescence imaging of the tumor vasculature. Their pharmacokinetics and tumor-targeting efficacy/specificity inmore » mice bearing murine breast 4T1 tumor were thoroughly investigated. In conclusion, ZnO NPs with dual-modality imaging properties can serve as an attractive candidate for future cancer theranostics.« less

Assessment of fibrotic liver disease with multimodal nonlinear optical microscopy

NASA Astrophysics Data System (ADS)

Lu, Fake; Zheng, Wei; Tai, Dean C. S.; Lin, Jian; Yu, Hanry; Huang, Zhiwei

2010-02-01

Liver fibrosis is the excessive accumulation of extracellular matrix proteins such as collagens, which may result in cirrhosis, liver failure, and portal hypertension. In this study, we apply a multimodal nonlinear optical microscopy platform developed to investigate the fibrotic liver diseases in rat models established by performing bile duct ligation (BDL) surgery. The three nonlinear microscopy imaging modalities are implemented on the same sectioned tissues of diseased model sequentially: i.e., second harmonic generation (SHG) imaging quantifies the contents of the collagens, the two-photon excitation fluorescence (TPEF) imaging reveals the morphology of hepatic cells, while coherent anti-Stokes Raman scattering (CARS) imaging maps the distributions of fats or lipids quantitatively across the tissue. Our imaging results show that during the development of liver fibrosis (collagens) in BDL model, fatty liver disease also occurs. The aggregated concentrations of collagen and fat constituents in liver fibrosis model show a certain correlationship between each other.

Multi-modality molecular imaging: pre-clinical laboratory configuration

NASA Astrophysics Data System (ADS)

Wu, Yanjun; Wellen, Jeremy W.; Sarkar, Susanta K.

2006-02-01

In recent years, the prevalence of in vivo molecular imaging applications has rapidly increased. Here we report on the construction of a multi-modality imaging facility in a pharmaceutical setting that is expected to further advance existing capabilities for in vivo imaging of drug distribution and the interaction with their target. The imaging instrumentation in our facility includes a microPET scanner, a four wavelength time-domain optical imaging scanner, a 9.4T/30cm MRI scanner and a SPECT/X-ray CT scanner. An electronics shop and a computer room dedicated to image analysis are additional features of the facility. The layout of the facility was designed with a central animal preparation room surrounded by separate laboratory rooms for each of the major imaging modalities to accommodate the work-flow of simultaneous in vivo imaging experiments. This report will focus on the design of and anticipated applications for our microPET and optical imaging laboratory spaces. Additionally, we will discuss efforts to maximize the daily throughput of animal scans through development of efficient experimental work-flows and the use of multiple animals in a single scanning session.

Quantitative reconstructions in multi-modal photoacoustic and optical coherence tomography imaging

NASA Astrophysics Data System (ADS)

Elbau, P.; Mindrinos, L.; Scherzer, O.

2018-01-01

In this paper we perform quantitative reconstruction of the electric susceptibility and the Grüneisen parameter of a non-magnetic linear dielectric medium using measurement of a multi-modal photoacoustic and optical coherence tomography system. We consider the mathematical model presented in Elbau et al (2015 Handbook of Mathematical Methods in Imaging ed O Scherzer (New York: Springer) pp 1169-204), where a Fredholm integral equation of the first kind for the Grüneisen parameter was derived. For the numerical solution of the integral equation we consider a Galerkin type method.

Detection of suspicious pain regions on a digital infrared thermal image using the multimodal function optimization.

PubMed

Lee, Junghoon; Lee, Joosung; Song, Sangha; Lee, Hyunsook; Lee, Kyoungjoung; Yoon, Youngro

2008-01-01

Automatic detection of suspicious pain regions is very useful in the medical digital infrared thermal imaging research area. To detect those regions, we use the SOFES (Survival Of the Fitness kind of the Evolution Strategy) algorithm which is one of the multimodal function optimization methods. We apply this algorithm to famous diseases, such as a foot of the glycosuria, the degenerative arthritis and the varicose vein. The SOFES algorithm is available to detect some hot spots or warm lines as veins. And according to a hundred of trials, the algorithm is very fast to converge.

Clinically Approved Nanoparticle Imaging Agents

PubMed Central

Thakor, Avnesh S.; Jokerst, Jesse V.; Ghanouni, Pejman; Campbell, Jos L.; Mittra, Erik

2016-01-01

Nanoparticles are a new class of imaging agent used for both anatomic and molecular imaging. Nanoparticle-based imaging exploits the signal intensity, stability, and biodistribution behavior of submicron-diameter molecular imaging agents. This review focuses on nanoparticles used in human medical imaging, with an emphasis on radionuclide imaging and MRI. Newer nanoparticle platforms are also discussed in relation to theranostic and multimodal uses. PMID:27738007

Transferring biomarker into molecular probe: Melanin nanoparticle as a naturally active platform for multimodality imaging

DOE PAGES

Fan, Quli; Cheng, Kai; Hu, Xiang; ...

2014-10-07

Developing multifunctional and easily prepared nanoplatforms with integrated different modalities is highly challenging for molecular imaging. Here, we report the successful transfer of an important molecular target, melanin, into a novel multimodality imaging nanoplatform. Melanin is abundantly expressed in melanotic melanomas and thus has been actively studied as a target for melanoma imaging. In our work, the multifunctional biopolymer nanoplatform based on ultrasmall (<10 nm) water-soluble melanin nanoparticle (MNP) was developed and showed unique photoacoustic property and natural binding ability with metal ions (for example, 64Cu 2+, Fe 3+). Therefore, MNP can serve not only as a photoacoustic contrast agent,more » but also as a nanoplatform for positron emission tomography (PET) and magnetic resonance imaging (MRI). Traditional passive nanoplatforms require complicated and time-consuming processes for prebuilding reporting moieties or chemical modifications using active groups to integrate different contrast properties into one entity. In comparison, utilizing functional biomarker melanin can greatly simplify the building process. We further conjugated α vβ 3 integrins, cyclic c(RGDfC) peptide, to MNPs to allow for U87MG tumor accumulation due to its targeting property combined with the enhanced permeability and retention (EPR) effect. As a result, the multimodal properties of MNPs demonstrate the high potential of endogenous materials with multifunctions as nanoplatforms for molecular theranostics and clinical translation.« less

Observation of Geometric Parametric Instability Induced by the Periodic Spatial Self-Imaging of Multimode Waves

NASA Astrophysics Data System (ADS)

Krupa, Katarzyna; Tonello, Alessandro; Barthélémy, Alain; Couderc, Vincent; Shalaby, Badr Mohamed; Bendahmane, Abdelkrim; Millot, Guy; Wabnitz, Stefan

2016-05-01

Spatiotemporal mode coupling in highly multimode physical systems permits new routes for exploring complex instabilities and forming coherent wave structures. We present here the first experimental demonstration of multiple geometric parametric instability sidebands, generated in the frequency domain through resonant space-time coupling, owing to the natural periodic spatial self-imaging of a multimode quasi-continuous-wave beam in a standard graded-index multimode fiber. The input beam was launched in the fiber by means of an amplified microchip laser emitting sub-ns pulses at 1064 nm. The experimentally observed frequency spacing among sidebands agrees well with analytical predictions and numerical simulations. The first-order peaks are located at the considerably large detuning of 123.5 THz from the pump. These results open the remarkable possibility to convert a near-infrared laser directly into a broad spectral range spanning visible and infrared wavelengths, by means of a single resonant parametric nonlinear effect occurring in the normal dispersion regime. As further evidence of our strong space-time coupling regime, we observed the striking effect that all of the different sideband peaks were carried by a well-defined and stable bell-shaped spatial profile.

[Isolated left ventricular non-compaction associated with Ebstein's anomaly. Multimodality non-invasive imaging for the assessment of congenital heart disease].

PubMed

Renilla, Alfredo; Santamarta, Elena; Corros, Cecilia; Martín, María; Barreiro, Manuel; de la Hera, Jesús

2013-01-01

To establish the etiology of heart failure in patients with congenital heart disease can be challenging. Multiple concomitant anomalies that can be missed after an initial diagnosis could be seen in these patients. In patients with congenital heart disease, a more accurate evaluation of cardiac morphology and left ventricular systolic function could be evaluated by recent non-invasive cardiac imaging techniques. We present a rare case where multimodal cardiac imaging was useful to establish the final diagnosis of left ventricular non-compaction associated with Ebstein's anomaly. Copyright © 2012 Instituto Nacional de Cardiología Ignacio Chávez. Published by Masson Doyma México S.A. All rights reserved.

Multimodal and synthetic aperture approach to full-field 3D shape and displacement measurements

NASA Astrophysics Data System (ADS)

Kujawińska, M.; Sitnik, R.

2017-08-01

Recently most of the measurement tasks in industry, civil engineering and culture heritage applications require archiving, characterization and monitoring of 3D objects and structures and their performance under changing conditions. These requirements can be met if multimodal measurement (MM) strategy is applied. It rely on effective combining structured light method and 3D digital image correlation with laser scanning/ToF, thermal imaging, multispectral imaging and BDRF measurements. In the case of big size and/or complicated objects MM have to be combined with hierarchical or synthetic aperture (SA) measurements. The new solutions in MM and SA strategies are presented and their applicability is shown at interesting cultural heritage and civil engineering applications.

Machine learning-based diagnosis of melanoma using macro images.

PubMed

Gautam, Diwakar; Ahmed, Mushtaq; Meena, Yogesh Kumar; Ul Haq, Ahtesham

2018-05-01

Cancer bears a poisoning threat to human society. Melanoma, the skin cancer, originates from skin layers and penetrates deep into subcutaneous layers. There exists an extensive research in melanoma diagnosis using dermatoscopic images captured through a dermatoscope. While designing a diagnostic model for general handheld imaging systems is an emerging trend, this article proposes a computer-aided decision support system for macro images captured by a general-purpose camera. General imaging conditions are adversely affected by nonuniform illumination, which further affects the extraction of relevant information. To mitigate it, we process an image to define a smooth illumination surface using the multistage illumination compensation approach, and the infected region is extracted using the proposed multimode segmentation method. The lesion information is numerated as a feature set comprising geometry, photometry, border series, and texture measures. The redundancy in feature set is reduced using information theory methods, and a classification boundary is modeled to distinguish benign and malignant samples using support vector machine, random forest, neural network, and fast discriminative mixed-membership-based naive Bayesian classifiers. Moreover, the experimental outcome is supported by hypothesis testing and boxplot representation for classification losses. The simulation results prove the significance of the proposed model that shows an improved performance as compared with competing arts. Copyright © 2017 John Wiley & Sons, Ltd.

Toward in vivo diagnosis of skin cancer using multimode imaging dermoscopy: (II) molecular mapping of highly pigmented lesions

NASA Astrophysics Data System (ADS)

Vasefi, Fartash; MacKinnon, Nicholas; Farkas, Daniel L.

2014-03-01

We have developed a multimode imaging dermoscope that combines polarization and hyperspectral imaging with a computationally rapid analytical model. This approach employs specific spectral ranges of visible and near infrared wavelengths for mapping the distribution of specific skin bio-molecules. This corrects for the melanin-hemoglobin misestimation common to other systems, without resorting to complex and computationally intensive tissue optical models that are prone to inaccuracies due to over-modeling. Various human skin measurements including a melanocytic nevus, and venous occlusion conditions were investigated and compared with other ratiometric spectral imaging approaches. Access to the broad range of hyperspectral data in the visible and near-infrared range allows our algorithm to flexibly use different wavelength ranges for chromophore estimation while minimizing melanin-hemoglobin optical signature cross-talk.

Visual tracking for multi-modality computer-assisted image guidance

NASA Astrophysics Data System (ADS)

Basafa, Ehsan; Foroughi, Pezhman; Hossbach, Martin; Bhanushali, Jasmine; Stolka, Philipp

2017-03-01

With optical cameras, many interventional navigation tasks previously relying on EM, optical, or mechanical guidance can be performed robustly, quickly, and conveniently. We developed a family of novel guidance systems based on wide-spectrum cameras and vision algorithms for real-time tracking of interventional instruments and multi-modality markers. These navigation systems support the localization of anatomical targets, support placement of imaging probe and instruments, and provide fusion imaging. The unique architecture - low-cost, miniature, in-hand stereo vision cameras fitted directly to imaging probes - allows for an intuitive workflow that fits a wide variety of specialties such as anesthesiology, interventional radiology, interventional oncology, emergency medicine, urology, and others, many of which see increasing pressure to utilize medical imaging and especially ultrasound, but have yet to develop the requisite skills for reliable success. We developed a modular system, consisting of hardware (the Optical Head containing the mini cameras) and software (components for visual instrument tracking with or without specialized visual features, fully automated marker segmentation from a variety of 3D imaging modalities, visual observation of meshes of widely separated markers, instant automatic registration, and target tracking and guidance on real-time multi-modality fusion views). From these components, we implemented a family of distinct clinical and pre-clinical systems (for combinations of ultrasound, CT, CBCT, and MRI), most of which have international regulatory clearance for clinical use. We present technical and clinical results on phantoms, ex- and in-vivo animals, and patients.

Delivery of ultrashort spatially focused pulses through a multimode fiber

NASA Astrophysics Data System (ADS)

Morales-Delgado, Edgar E.; Papadopoulos, Ioannis N.; Farahi, Salma; Psaltis, Demetri; Moser, Christophe

2015-08-01

Multimode optical fibers potentially allow the transmission of larger amounts of information than their single mode counterparts because of their high number of supported modes. However, propagation of a light pulse through a multimode fiber suffers from spatial distortions due to the superposition of the various exited modes and from time broadening due to modal dispersion. We present a method based on digital phase conjugation to selectively excite in a multimode fiber specific optical fiber modes that follow similar optical paths as they travel through the fiber. The excited modes interfere constructively at the fiber output generating an ultrashort spatially focused pulse. The excitation of a limited number of modes following similar optical paths limits modal dispersion, allowing the transmission of the ultrashort pulse. We have experimentally demonstrated the delivery of a focused spot of pulse width equal to 500 fs through a 30 cm, 200 micrometer core step index multimode fiber. The results of this study show that two-photon imaging capability can be added to ultra-thin lensless endoscopy using commercial multimode fibers.

Delivery of an ultrashort spatially focused pulse to the other end of a multimode fiber using digital phase conjugation

NASA Astrophysics Data System (ADS)

Morales Delgado, Edgar E.; Papadopoulos, Ioannis N.; Farahi, Salma; Psaltis, Demetri; Moser, Christophe

2015-03-01

Multimode optical fibers potentially allow the transmission of larger amounts of information than their single mode counterparts because of their high number of supported modes. However, propagation of a light pulse through a multimode fiber suffers from spatial distortions due to the superposition of the various exited modes and from time broadening due to modal dispersion. We present a method based on digital phase conjugation to selectively excite in a multimode fiber specific optical fiber modes that follow similar optical paths as they travel through the fiber. The excited modes interfere constructively at the fiber output generating an ultrashort spatially focused pulse. The excitation of a limited number of modes following similar optical paths limits modal dispersion, allowing the transmission of the ultrashort pulse. We have experimentally demonstrated the delivery of a focused spot of pulse width equal to 500 fs through a 30 cm, 200 micrometer core step-index multimode fiber. The results of this study show that two-photon imaging capability can be added to ultra-thin lensless endoscopy using commercial multimode fibers.

Robust temporal alignment of multimodal cardiac sequences

NASA Astrophysics Data System (ADS)

Perissinotto, Andrea; Queirós, Sandro; Morais, Pedro; Baptista, Maria J.; Monaghan, Mark; Rodrigues, Nuno F.; D'hooge, Jan; Vilaça, João. L.; Barbosa, Daniel

2015-03-01

Given the dynamic nature of cardiac function, correct temporal alignment of pre-operative models and intraoperative images is crucial for augmented reality in cardiac image-guided interventions. As such, the current study focuses on the development of an image-based strategy for temporal alignment of multimodal cardiac imaging sequences, such as cine Magnetic Resonance Imaging (MRI) or 3D Ultrasound (US). First, we derive a robust, modality-independent signal from the image sequences, estimated by computing the normalized cross-correlation between each frame in the temporal sequence and the end-diastolic frame. This signal is a resembler for the left-ventricle (LV) volume curve over time, whose variation indicates different temporal landmarks of the cardiac cycle. We then perform the temporal alignment of these surrogate signals derived from MRI and US sequences of the same patient through Dynamic Time Warping (DTW), allowing to synchronize both sequences. The proposed framework was evaluated in 98 patients, which have undergone both 3D+t MRI and US scans. The end-systolic frame could be accurately estimated as the minimum of the image-derived surrogate signal, presenting a relative error of 1.6 +/- 1.9% and 4.0 +/- 4.2% for the MRI and US sequences, respectively, thus supporting its association with key temporal instants of the cardiac cycle. The use of DTW reduces the desynchronization of the cardiac events in MRI and US sequences, allowing to temporally align multimodal cardiac imaging sequences. Overall, a generic, fast and accurate method for temporal synchronization of MRI and US sequences of the same patient was introduced. This approach could be straightforwardly used for the correct temporal alignment of pre-operative MRI information and intra-operative US images.

Prussian blue nanocubes: multi-functional nanoparticles for multimodal imaging and image-guided therapy (Conference Presentation)

NASA Astrophysics Data System (ADS)

Cook, Jason R.; Dumani, Diego S.; Kubelick, Kelsey P.; Luci, Jeffrey; Emelianov, Stanislav Y.

2017-03-01

Imaging modalities utilize contrast agents to improve morphological visualization and to assess functional and molecular/cellular information. Here we present a new type of nanometer scale multi-functional particle that can be used for multi-modal imaging and therapeutic applications. Specifically, we synthesized monodisperse 20 nm Prussian Blue Nanocubes (PBNCs) with desired optical absorption in the near-infrared region and superparamagnetic properties. PBNCs showed excellent contrast in photoacoustic (700 nm wavelength) and MR (3T) imaging. Furthermore, photostability was assessed by exposing the PBNCs to nearly 1,000 laser pulses (5 ns pulse width) with up to 30 mJ/cm2 laser fluences. The PBNCs exhibited insignificant changes in photoacoustic signal, demonstrating enhanced robustness compared to the commonly used gold nanorods (substantial photodegradation with fluences greater than 5 mJ/cm2). Furthermore, the PBNCs exhibited superparamagnetism with a magnetic saturation of 105 emu/g, a 5x improvement over superparamagnetic iron-oxide (SPIO) nanoparticles. PBNCs exhibited enhanced T2 contrast measured using 3T clinical MRI. Because of the excellent optical absorption and magnetism, PBNCs have potential uses in other imaging modalities including optical tomography, microscopy, magneto-motive OCT/ultrasound, etc. In addition to multi-modal imaging, the PBNCs are multi-functional and, for example, can be used to enhance magnetic delivery and as therapeutic agents. Our initial studies show that stem cells can be labeled with PBNCs to perform image-guided magnetic delivery. Overall, PBNCs can act as imaging/therapeutic agents in diverse applications including cancer, cardiovascular disease, ophthalmology, and tissue engineering. Furthermore, PBNCs are based on FDA approved Prussian Blue thus potentially easing clinical translation of PBNCs.

STRUCTURAL AND FUNCTIONAL CHARACTERIZATION OF BENIGN FLECK RETINA USING MULTIMODAL IMAGING.

PubMed

Neriyanuri, Srividya; Rao, Chetan; Raman, Rajiv

2017-01-01

To report structural and functional features in a case series of benign fleck retina using multimodal imaging. Four cases with benign fleck retina underwent complete ophthalmic examination that included detailed history, visual acuity, and refractive error testing, FM-100 hue test, dilated fundus evaluation, full field electroretinogram, fundus photography with autofluorescence, fundus fluorescein angiography, and swept-source optical coherence tomography. Age group of the cases ranged from 19 years to 35 years (3 males and 1 female). Parental consanguinity was reported in two cases. All of them were visually asymptomatic with best-corrected visual acuity of 20/20 (moderate astigmatism) in both the eyes. Low color discrimination was seen in two cases. Fundus photography showed pisciform flecks which were compactly placed on posterior pole and were discrete, diverging towards periphery. Lesions were seen as smaller dots within 1500 microns from fovea and were hyperfluorescent on autofluorescence. Palisading retinal pigment epithelium defects were seen in posterior pole on fundus fluorescein angiography imaging; irregular hyper fluorescence was also noted. One case had reduced cone responses on full field electroretinogram; the other three cases had normal electroretinogram. On optical coherence tomography, level of lesions varied from retinal pigment epithelium, inner segment to outer segment extending till external limiting membrane. Functional and structural deficits in benign fleck retina were picked up using multimodal imaging.

Near-infrared light-triggered theranostics for tumor-specific enhanced multimodal imaging and photothermal therapy

PubMed Central

Wu, Bo; Wan, Bing; Lu, Shu-Ting; Deng, Kai; Li, Xiao-Qi; Wu, Bao-Lin; Li, Yu-Shuang; Liao, Ru-Fang; Huang, Shi-Wen; Xu, Hai-Bo

2017-01-01

The major challenge in current clinic contrast agents (CAs) and chemotherapy is the poor tumor selectivity and response. Based on the self-quench property of IR820 at high concentrations, and different contrast effect ability of Gd-DOTA between inner and outer of liposome, we developed “bomb-like” light-triggered CAs (LTCAs) for enhanced CT/MRI/FI multimodal imaging, which can improve the signal-to-noise ratio of tumor tissue specifically. IR820, Iohexol and Gd-chelates were firstly encapsulated into the thermal-sensitive nanocarrier with a high concentration. This will result in protection and fluorescence quenching. Then, the release of CAs was triggered by near-infrared (NIR) light laser irradiation, which will lead to fluorescence and MRI activation and enable imaging of inflammation. In vitro and in vivo experiments demonstrated that LTCAs with 808 nm laser irradiation have shorter T1 relaxation time in MRI and stronger intensity in FI compared to those without irradiation. Additionally, due to the high photothermal conversion efficiency of IR820, the injection of LTCAs was demonstrated to completely inhibit C6 tumor growth in nude mice up to 17 days after NIR laser irradiation. The results indicate that the LTCAs can serve as a promising platform for NIR-activated multimodal imaging and photothermal therapy. PMID:28670120

SU-E-I-23: Design and Clinical Application of External Marking Body in Multi- Mode Medical Images Registration and Fusion

DOE Office of Scientific and Technical Information (OSTI.GOV)

Chen, Z; Gong, G

2014-06-01

Purpose: To design an external marking body (EMB) that could be visible on computed tomography (CT), magnetic resonance (MR), positron emission tomography (PET) and single-photon emission computed tomography (SPECT) images and to investigate the use of the EMB for multiple medical images registration and fusion in the clinic. Methods: We generated a solution containing paramagnetic metal ions and iodide ions (CT'MR dual-visible solution) that could be viewed on CT and MR images and multi-mode image visible solution (MIVS) that could be obtained by mixing radioactive nuclear material. A globular plastic theca (diameter: 3–6 mm) that mothball the MIVS and themore » EMB was brought by filling MIVS. The EMBs were fixed on the patient surface and CT, MR, PET and SPECT scans were obtained. The feasibility of clinical application and the display and registration error of EMB among different image modalities were investigated. Results: The dual-visible solution was highly dense on CT images (HU>700). A high signal was also found in all MR scanning (T1, T2, STIR and FLAIR) images, and the signal was higher than subcutaneous fat. EMB with radioactive nuclear material caused a radionuclide concentration area on PET and SPECT images, and the signal of EMB was similar to or higher than tumor signals. The theca with MIVS was clearly visible on all the images without artifact, and the shape was round or oval with a sharp edge. The maximum diameter display error was 0.3 ± 0.2mm on CT and MRI images, and 1.0 ± 0.3mm on PET and SPECT images. In addition, the registration accuracy of the theca center among multi-mode images was less than 1mm. Conclusion: The application of EMB with MIVS improves the registration and fusion accuracy of multi-mode medical images. Furthermore, it has the potential to ameliorate disease diagnosis and treatment outcome.« less

Stent-induced coronary artery stenosis characterized by multimodal nonlinear optical microscopy

NASA Astrophysics Data System (ADS)

Wang, Han-Wei; Simianu, Vlad; Locker, Mattew J.; Cheng, Ji-Xin; Sturek, Michael

2011-02-01

We demonstrate for the first time the applicability of multimodal nonlinear optical (NLO) microscopy to the interrogation of stented coronary arteries under different diet and stent deployment conditions. Bare metal stents and Taxus drug-eluting stents (DES) were placed in coronary arteries of Ossabaw pigs of control and atherogenic diet groups. Multimodal NLO imaging was performed to inspect changes in arterial structures and compositions after stenting. Sum frequency generation, one of the multimodalities, was used for the quantitative analysis of collagen content in the peristent and in-stent artery segments of both pig groups. Atherogenic diet increased lipid and collagen in peristent segments. In-stent segments showed decreased collagen expression in neointima compared to media. Deployment of DES in atheromatous arteries inhibited collagen expression in the arterial media.

PET/MRI assessment of the infarcted mouse heart

NASA Astrophysics Data System (ADS)

Buonincontri, Guido; Methner, Carmen; Krieg, Thomas; Hawkes, Robert C.; Adrian Carpenter, T.; Sawiak, Stephen J.

2014-01-01

Heart failure originating from myocardial infarction (MI) is a leading cause of death worldwide. Mouse models of ischaemia and reperfusion injury (I/R) are used to study the effects of novel treatment strategies targeting MI, however staging disease and treatment efficacy is a challenge. Damage and recovery can be assessed on the cellular, tissue or whole-organ scale but these are rarely measured in concert. Here, for the first time, we present data showing measures of injury in infarcted mice using complementary techniques for multi-modal characterisation of the heart. We use in vivo magnetic resonance imaging (MRI) to assess heart function with cine-MRI, hindered perfusion with late gadolinium enhancement imaging and muscular function with displacement encoded with stimulated echoes (DENSE) MRI. These measures are followed by positron emission tomography (PET) with 18-F-fluorodeoxyglucose to assess cellular metabolism. We demonstrate a protocol combining each of these measures for the same animal in the same imaging session and compare how the different markers can be used to quantify cardiac recovery on different scales following injury.

Quantitative label-free multimodality nonlinear optical imaging for in situ differentiation of cancerous lesions

NASA Astrophysics Data System (ADS)

Xu, Xiaoyun; Li, Xiaoyan; Cheng, Jie; Liu, Zhengfan; Thrall, Michael J.; Wang, Xi; Wang, Zhiyong; Wong, Stephen T. C.

2013-03-01

The development of real-time, label-free imaging techniques has recently attracted research interest for in situ differentiation of cancerous lesions from normal tissues. Molecule-specific intrinsic contrast can arise from label-free imaging techniques such as Coherent Anti-Stokes Raman Scattering (CARS), Two-Photon Excited AutoFluorescence (TPEAF), and Second Harmonic Generation (SHG), which, in combination, would hold the promise of a powerful label-free tool for cancer diagnosis. Among cancer-related deaths, lung carcinoma is the leading cause for both sexes. Although early treatment can increase the survival rate dramatically, lesion detection and precise diagnosis at an early stage is unusual due to its asymptomatic nature and limitations of current diagnostic techniques that make screening difficult. We investigated the potential of using multimodality nonlinear optical microscopy that incorporates CARS, TPEAF, and SHG techniques for differentiation of lung cancer from normal tissue. Cancerous and non-cancerous lung tissue samples from patients were imaged using CARS, TPEAF, and SHG techniques for comparison. These images showed good pathology correlation with hematoxylin and eosin (H and E) stained sections from the same tissue samples. Ongoing work includes imaging at various penetration depths to show three-dimensional morphologies of tumor cell nuclei using CARS, elastin using TPEAF, and collagen using SHG and developing classification algorithms for quantitative feature extraction to enable lung cancer diagnosis. Our results indicate that via real-time morphology analyses, a multimodality nonlinear optical imaging platform potentially offers a powerful minimally-invasive way to differentiate cancer lesions from surrounding non-tumor tissues in vivo for clinical applications.

Intra-operative label-free multimodal multiphoton imaging of breast cancer margins and microenvironment (Conference Presentation)

NASA Astrophysics Data System (ADS)

Sun, Yi; You, Sixian; Tu, Haohua; Spillman, Darold R.; Marjanovic, Marina; Chaney, Eric J.; Liu, George Z.; Ray, Partha S.; Higham, Anna; Boppart, Stephen A.

2017-02-01

Label-free multi-photon imaging has been a powerful tool for studying tissue microstructures and biochemical distributions, particularly for investigating tumors and their microenvironments. However, it remains challenging for traditional bench-top multi-photon microscope systems to conduct ex vivo tumor tissue imaging in the operating room due to their bulky setups and laser sources. In this study, we designed, built, and clinically demonstrated a portable multi-modal nonlinear label-free microscope system that combined four modalities, including two- and three- photon fluorescence for studying the distributions of FAD and NADH, and second and third harmonic generation, respectively, for collagen fiber structures and the distribution of micro-vesicles found in tumors and the microenvironment. Optical realignments and switching between modalities were motorized for more rapid and efficient imaging and for a light-tight enclosure, reducing ambient light noise to only 5% within the brightly lit operating room. Using up to 20 mW of laser power after a 20x objective, this system can acquire multi-modal sets of images over 600 μm × 600 μm at an acquisition rate of 60 seconds using galvo-mirror scanning. This portable microscope system was demonstrated in the operating room for imaging fresh, resected, unstained breast tissue specimens, and for assessing tumor margins and the tumor microenvironment. This real-time label-free nonlinear imaging system has the potential to uniquely characterize breast cancer margins and the microenvironment of tumors to intraoperatively identify structural, functional, and molecular changes that could indicate the aggressiveness of the tumor.

Investigating the Abscopal Effects of Radioablation on Shielded Bone Marrow in Rodent Models Using Multimodality Imaging.

PubMed

Afshar, Solmaz F; Zawaski, Janice A; Inoue, Taeko; Rendon, David A; Zieske, Arthur W; Punia, Jyotinder N; Sabek, Omaima M; Gaber, M Waleed

2017-07-01

The abscopal effect is the response to radiation at sites that are distant from the irradiated site of an organism, and it is thought to play a role in bone marrow (BM) recovery by initiating responses in the unirradiated bone marrow. Understanding the mechanism of this effect has applications in treating BM failure (BMF) and BM transplantation (BMT), and improving survival of nuclear disaster victims. Here, we investigated the use of multimodality imaging as a translational tool to longitudinally assess bone marrow recovery. We used positron emission tomography/computed tomography (PET/CT), magnetic resonance imaging (MRI) and optical imaging to quantify bone marrow activity, vascular response and marrow repopulation in fully and partially irradiated rodent models. We further measured the effects of radiation on serum cytokine levels, hematopoietic cell counts and histology. PET/CT imaging revealed a radiation-induced increase in proliferation in the shielded bone marrow (SBM) compared to exposed bone marrow (EBM) and sham controls. T 2 -weighted MRI showed radiation-induced hemorrhaging in the EBM and unirradiated SBM. In the EBM and SBM groups, we found alterations in serum cytokine and hormone levels and in hematopoietic cell population proportions, and histological evidence of osteoblast activation at the bone marrow interface. Importantly, we generated a BMT mouse model using fluorescent-labeled bone marrow donor cells and performed fluorescent imaging to reveal the migration of bone marrow cells from shielded to radioablated sites. Our study validates the use of multimodality imaging to monitor bone marrow recovery and provides evidence for the abscopal response in promoting bone marrow recovery after irradiation.

Folic acid-targeted magnetic Tb-doped CeF3 fluorescent nanoparticles as bimodal probes for cellular fluorescence and magnetic resonance imaging.

PubMed

Ma, Zhi-Ya; Liu, Yu-Ping; Bai, Ling-Yu; An, Jie; Zhang, Lin; Xuan, Yang; Zhang, Xiao-Shuai; Zhao, Yuan-Di

2015-10-07

Magnetic fluorescent nanoparticles (NPs) have great potential applications for diagnostics, imaging and therapy. We developed a facile polyol method to synthesize multifunctional Fe3O4@CeF3:Tb@CeF3 NPs with small size (<20 nm), high water solubility and good biocompatibility. The NPs were modified by ligand exchange reactions with citric acid (CA) to obtain carboxyl-functionalized NPs (Fe3O4@CeF3:Tb@CeF3-COOH). Folic acid (FA) as an affinity ligand was then covalently conjugated onto NPs to yield Fe3O4@CeF3:Tb@CeF3-FA NPs. They were then applied as multimodal imaging agents for simultaneous in vitro targeted fluorescence imaging and magnetic resonance imaging (MRI) of HeLa cells with overexpressed folate receptors (FR). The results indicated that these NPs had strong luminescence and enhanced T2-weighted MR contrast and would be promising candidates as multimodal probes for both fluorescence and MRI imaging.

Predictive assessment of kidney functional recovery following ischemic injury using optical spectroscopy

DOE Office of Scientific and Technical Information (OSTI.GOV)

Raman, Rajesh N.; Pivetti, Christopher D.; Ramsamooj, Rajendra

Functional changes in rat kidneys during the induced ischemic injury and recovery phases were explored using multimodal autofluorescence and light scattering imaging. We aim to evaluate the use of noncontact optical signatures for rapid assessment of tissue function and viability. Specifically, autofluorescence images were acquired in vivo under 355, 325, and 266 nm illumination while light scattering images were collected at the excitation wavelengths as well as using relatively narrowband light centered at 500 nm. The images were simultaneously recorded using a multimodal optical imaging system. We also analyzed to obtain time constants, which were correlated to kidney dysfunction asmore » determined by a subsequent survival study and histopathological analysis. This analysis of both the light scattering and autofluorescence images suggests that changes in tissue microstructure, fluorophore emission, and blood absorption spectral characteristics, coupled with vascular response, contribute to the behavior of the observed signal, which may be used to obtain tissue functional information and offer the ability to predict posttransplant kidney function.« less

A versatile clearing agent for multi-modal brain imaging

PubMed Central

Costantini, Irene; Ghobril, Jean-Pierre; Di Giovanna, Antonino Paolo; Mascaro, Anna Letizia Allegra; Silvestri, Ludovico; Müllenbroich, Marie Caroline; Onofri, Leonardo; Conti, Valerio; Vanzi, Francesco; Sacconi, Leonardo; Guerrini, Renzo; Markram, Henry; Iannello, Giulio; Pavone, Francesco Saverio

2015-01-01

Extensive mapping of neuronal connections in the central nervous system requires high-throughput µm-scale imaging of large volumes. In recent years, different approaches have been developed to overcome the limitations due to tissue light scattering. These methods are generally developed to improve the performance of a specific imaging modality, thus limiting comprehensive neuroanatomical exploration by multi-modal optical techniques. Here, we introduce a versatile brain clearing agent (2,2′-thiodiethanol; TDE) suitable for various applications and imaging techniques. TDE is cost-efficient, water-soluble and low-viscous and, more importantly, it preserves fluorescence, is compatible with immunostaining and does not cause deformations at sub-cellular level. We demonstrate the effectiveness of this method in different applications: in fixed samples by imaging a whole mouse hippocampus with serial two-photon tomography; in combination with CLARITY by reconstructing an entire mouse brain with light sheet microscopy and in translational research by imaging immunostained human dysplastic brain tissue. PMID:25950610

Predictive assessment of kidney functional recovery following ischemic injury using optical spectroscopy

DOE PAGES

Raman, Rajesh N.; Pivetti, Christopher D.; Ramsamooj, Rajendra; ...

2017-05-03

Functional changes in rat kidneys during the induced ischemic injury and recovery phases were explored using multimodal autofluorescence and light scattering imaging. We aim to evaluate the use of noncontact optical signatures for rapid assessment of tissue function and viability. Specifically, autofluorescence images were acquired in vivo under 355, 325, and 266 nm illumination while light scattering images were collected at the excitation wavelengths as well as using relatively narrowband light centered at 500 nm. The images were simultaneously recorded using a multimodal optical imaging system. We also analyzed to obtain time constants, which were correlated to kidney dysfunction asmore » determined by a subsequent survival study and histopathological analysis. This analysis of both the light scattering and autofluorescence images suggests that changes in tissue microstructure, fluorophore emission, and blood absorption spectral characteristics, coupled with vascular response, contribute to the behavior of the observed signal, which may be used to obtain tissue functional information and offer the ability to predict posttransplant kidney function.« less

Adaptive Optics Imaging in Laser Pointer Maculopathy.

PubMed

Sheyman, Alan T; Nesper, Peter L; Fawzi, Amani A; Jampol, Lee M

2016-08-01

The authors report multimodal imaging including adaptive optics scanning laser ophthalmoscopy (AOSLO) (Apaeros retinal image system AOSLO prototype; Boston Micromachines Corporation, Boston, MA) in a case of previously diagnosed unilateral acute idiopathic maculopathy (UAIM) that demonstrated features of laser pointer maculopathy. The authors also show the adaptive optics images of a laser pointer maculopathy case previously reported. A 15-year-old girl was referred for the evaluation of a maculopathy suspected to be UAIM. The authors reviewed the patient's history and obtained fluorescein angiography, autofluorescence, optical coherence tomography, infrared reflectance, and AOSLO. The time course of disease and clinical examination did not fit with UAIM, but the linear pattern of lesions was suspicious for self-inflicted laser pointer injury. This was confirmed on subsequent questioning of the patient. The presence of linear lesions in the macula that are best highlighted with multimodal imaging techniques should alert the physician to the possibility of laser pointer injury. AOSLO further characterizes photoreceptor damage in this condition. [Ophthalmic Surg Lasers Imaging Retina. 2016;47:782-785.]. Copyright 2016, SLACK Incorporated.

Multimodal Fusion with Reference: Searching for Joint Neuromarkers of Working Memory Deficits in Schizophrenia

PubMed Central

Qi, Shile; Calhoun, Vince D.; van Erp, Theo G. M.; Bustillo, Juan; Damaraju, Eswar; Turner, Jessica A.; Du, Yuhui; Chen, Jiayu; Yu, Qingbao; Mathalon, Daniel H.; Ford, Judith M.; Voyvodic, James; Mueller, Bryon A.; Belger, Aysenil; Ewen, Sarah Mc; Potkin, Steven G.; Preda, Adrian; Jiang, Tianzi

2017-01-01

Multimodal fusion is an effective approach to take advantage of cross-information among multiple imaging data to better understand brain diseases. However, most current fusion approaches are blind, without adopting any prior information. To date, there is increasing interest to uncover the neurocognitive mapping of specific behavioral measurement on enriched brain imaging data; hence, a supervised, goal-directed model that enables a priori information as a reference to guide multimodal data fusion is in need and a natural option. Here we proposed a fusion with reference model, called “multi-site canonical correlation analysis with reference plus joint independent component analysis” (MCCAR+jICA), which can precisely identify co-varying multimodal imaging patterns closely related to reference information, such as cognitive scores. In a 3-way fusion simulation, the proposed method was compared with its alternatives on estimation accuracy of both target component decomposition and modality linkage detection. MCCAR+jICA outperforms others with higher precision. In human imaging data, working memory performance was utilized as a reference to investigate the covarying functional and structural brain patterns among 3 modalities and how they are impaired in schizophrenia. Two independent cohorts (294 and 83 subjects respectively) were used. Interestingly, similar brain maps were identified between the two cohorts, with substantial overlap in the executive control networks in fMRI, salience network in sMRI, and major white matter tracts in dMRI. These regions have been linked with working memory deficits in schizophrenia in multiple reports, while MCCAR+jICA further verified them in a repeatable, joint manner, demonstrating the potential of such results to identify potential neuromarkers for mental disorders. PMID:28708547

MULTIMODAL IMAGING ADDS NEW INSIGHTS INTO ACUTE SYPHILITIC POSTERIOR PLACOID CHORIORETINITIS.

PubMed

Tsui, Edmund; Gal-Or, Orly; Ghadiali, Quraish; Freund, K Bailey

2017-10-11

Acute syphilitic posterior placoid chorioretinitis (ASPPC) is an uncommon manifestation of ocular syphilis with distinct clinical features. We describe new multimodal imaging findings in a patient with ASPPC. Observational case report with multimodal imaging. A 44-year-old woman presented with 5 days of decreased vision in her right eye. Visual acuity was counting fingers in her right eye and 20/20 in her left eye. Funduscopic examination of the right eye showed a yellow placoid macular lesion with extension beyond the equator, which was encircled by an annular ring of outer retinal whitening. Ultra-widefield fundus autofluorescence demonstrated hyperautofluorescence corresponding to the placoid lesion. Examination of the left eye appeared unremarkable, but ultra-widefield fundus autofluorescence showed an area of hyperautofluorescence located superonasal to the optic nerve. Optical coherence tomography of the right eye demonstrated subretinal fluid and overlying disruption of the ellipsoid zone. Fluorescein angiography demonstrated early hypofluorescent and hyperfluorescent spots and late staining within the placoid lesion. Optical coherence tomography angiography showed several areas of decreased flow signal within the placoid lesion at the level of the choriocapillaris. Laboratory testing revealed a rapid plasma reagin titer of 1:1,024. Two months after treatment with intravenous penicillin G, visual acuity had improved to 20/25 in her right eye, and optical coherence tomography showed partial restoration of the ellipsoid zone. The annular ring resolved with near normalization of fundus autofluorescence and optical coherence tomography angiography demonstrated resolution of flow. Multimodal imaging provides further insight into the pathogenesis of ASPPC. Ultra-widefield fundus autofluorescence may show evidence of ellipsoid zone disruption in areas that clinically appear normal. Flow voids within the choriocapillaris in ASPPC appear to resolve with appropriate treatment, a finding that suggests a transient disruption of choriocapillaris flow in ASPPC.

Nanoparticles in practice for molecular-imaging applications: An overview.

PubMed

Padmanabhan, Parasuraman; Kumar, Ajay; Kumar, Sundramurthy; Chaudhary, Ravi Kumar; Gulyás, Balázs

2016-09-01

Nanoparticles (NPs) are playing a progressively more significant role in multimodal and multifunctional molecular imaging. The agents like Superparamagnetic iron oxide (SPIO), manganese oxide (MnO), gold NPs/nanorods and quantum dots (QDs) possess specific properties like paramagnetism, superparamagnetism, surface plasmon resonance (SPR) and photoluminescence respectively. These specific properties make them able for single/multi-modal and single/multi-functional molecular imaging. NPs generally have nanomolar or micromolar sensitivity range and can be detected via imaging instrumentation. The distinctive characteristics of these NPs make them suitable for imaging, therapy and delivery of drugs. Multifunctional nanoparticles (MNPs) can be produced through either modification of shell or surface or by attaching an affinity ligand to the nanoparticles. They are utilized for targeted imaging by magnetic resonance imaging (MRI), single photon emission computed tomography (SPECT), positron emission tomography (PET), computed tomography (CT), photo acoustic imaging (PAI), two photon or fluorescent imaging and ultra sound etc. Toxicity factor of NPs is also a very important concern and toxic effect should be eliminated. First generation NPs have been designed, developed and tested in living subjects and few of them are already in clinical use. In near future, molecular imaging will get advanced with multimodality and multifunctionality to detect diseases like cancer, neurodegenerative diseases, cardiac diseases, inflammation, stroke, atherosclerosis and many others in their early stages. In the current review, we discussed single/multifunctional nanoparticles along with molecular imaging modalities. The present article intends to reveal recent avenues for nanomaterials in multimodal and multifunctional molecular imaging through a review of pertinent literatures. The topic emphasises on the distinctive characteristics of nanomaterial which makes them, suitable for biomedical imaging, therapy and delivery of drugs. This review is more informative of indicative technologies which will be helpful in a way to plan, understand and lead the nanotechnology related work. Copyright © 2016 Acta Materialia Inc. Published by Elsevier Ltd. All rights reserved.

Use of multidimensional, multimodal imaging and PACS to support neurological diagnoses

DOE Office of Scientific and Technical Information (OSTI.GOV)

Wong, S.T.C.; Knowlton, R.; Hoo, K.S.

1995-12-31

Technological advances in brain imaging have revolutionized diagnosis in neurology and neurological surgery. Major imaging techniques include magnetic resonance imaging (MRI) to visualize structural anatomy, positron emission tomography (PET) to image metabolic function and cerebral blood flow, magnetoencephalography (MEG) to visualize the location of physiologic current sources, and magnetic resonance spectroscopy (MRS) to measure specific biochemicals. Each of these techniques studies different biomedical aspects of the grain, but there lacks an effective means to quantify and correlate the disparate imaging datasets in order to improve clinical decision making processes. This paper describes several techniques developed in a UNIX-based neurodiagnostic workstationmore » to aid the non-invasive presurgical evaluation of epilepsy patients. These techniques include on-line access to the picture archiving and communication systems (PACS) multimedia archive, coregistration of multimodality image datasets, and correlation and quantitative of structural and functional information contained in the registered images. For illustration, the authors describe the use of these techniques in a patient case of non-lesional neocortical epilepsy. They also present the future work based on preliminary studies.« less

Dermatological Feasibility of Multimodal Facial Color Imaging Modality for Cross-Evaluation of Facial Actinic Keratosis

PubMed Central

Bae, Youngwoo; Son, Taeyoon; Nelson, J. Stuart; Kim, Jae-Hong; Choi, Eung Ho; Jung, Byungjo

2010-01-01

Background/Purpose Digital color image analysis is currently considered as a routine procedure in dermatology. In our previous study, a multimodal facial color imaging modality (MFCIM), which provides a conventional, parallel- and cross-polarization, and fluorescent color image, was introduced for objective evaluation of various facial skin lesions. This study introduces a commercial version of MFCIM, DermaVision-PRO, for routine clinical use in dermatology and demonstrates its dermatological feasibility for cross-evaluation of skin lesions. Methods/Results Sample images of subjects with actinic keratosis or non-melanoma skin cancers were obtained at four different imaging modes. Various image analysis methods were applied to cross-evaluate the skin lesion and, finally, extract valuable diagnostic information. DermaVision-PRO is potentially a useful tool as an objective macroscopic imaging modality for quick prescreening and cross-evaluation of facial skin lesions. Conclusion DermaVision-PRO may be utilized as a useful tool for cross-evaluation of widely distributed facial skin lesions and an efficient database management of patient information. PMID:20923462

Multi-modality imaging of tumor phenotype and response to therapy

NASA Astrophysics Data System (ADS)

Nyflot, Matthew J.

2011-12-01

Imaging and radiation oncology have historically been closely linked. However, the vast majority of techniques used in the clinic involve anatomical imaging. Biological imaging offers the potential for innovation in the areas of cancer diagnosis and staging, radiotherapy target definition, and treatment response assessment. Some relevant imaging techniques are FDG PET (for imaging cellular metabolism), FLT PET (proliferation), CuATSM PET (hypoxia), and contrast-enhanced CT (vasculature and perfusion). Here, a technique for quantitative spatial correlation of tumor phenotype is presented for FDG PET, FLT PET, and CuATSM PET images. Additionally, multimodality imaging of treatment response with FLT PET, CuATSM, and dynamic contrast-enhanced CT is presented, in a trial of patients receiving an antiangiogenic agent (Avastin) combined with cisplatin and radiotherapy. Results are also presented for translational applications in animal models, including quantitative assessment of proliferative response to cetuximab with FLT PET and quantification of vascular volume with a blood-pool contrast agent (Fenestra). These techniques have clear applications to radiobiological research and optimized treatment strategies, and may eventually be used for personalized therapy for patients.

VoxelStats: A MATLAB Package for Multi-Modal Voxel-Wise Brain Image Analysis.

PubMed

Mathotaarachchi, Sulantha; Wang, Seqian; Shin, Monica; Pascoal, Tharick A; Benedet, Andrea L; Kang, Min Su; Beaudry, Thomas; Fonov, Vladimir S; Gauthier, Serge; Labbe, Aurélie; Rosa-Neto, Pedro

2016-01-01

In healthy individuals, behavioral outcomes are highly associated with the variability on brain regional structure or neurochemical phenotypes. Similarly, in the context of neurodegenerative conditions, neuroimaging reveals that cognitive decline is linked to the magnitude of atrophy, neurochemical declines, or concentrations of abnormal protein aggregates across brain regions. However, modeling the effects of multiple regional abnormalities as determinants of cognitive decline at the voxel level remains largely unexplored by multimodal imaging research, given the high computational cost of estimating regression models for every single voxel from various imaging modalities. VoxelStats is a voxel-wise computational framework to overcome these computational limitations and to perform statistical operations on multiple scalar variables and imaging modalities at the voxel level. VoxelStats package has been developed in Matlab(®) and supports imaging formats such as Nifti-1, ANALYZE, and MINC v2. Prebuilt functions in VoxelStats enable the user to perform voxel-wise general and generalized linear models and mixed effect models with multiple volumetric covariates. Importantly, VoxelStats can recognize scalar values or image volumes as response variables and can accommodate volumetric statistical covariates as well as their interaction effects with other variables. Furthermore, this package includes built-in functionality to perform voxel-wise receiver operating characteristic analysis and paired and unpaired group contrast analysis. Validation of VoxelStats was conducted by comparing the linear regression functionality with existing toolboxes such as glim_image and RMINC. The validation results were identical to existing methods and the additional functionality was demonstrated by generating feature case assessments (t-statistics, odds ratio, and true positive rate maps). In summary, VoxelStats expands the current methods for multimodal imaging analysis by allowing the estimation of advanced regional association metrics at the voxel level.

A graph-based approach for the retrieval of multi-modality medical images.

PubMed

Kumar, Ashnil; Kim, Jinman; Wen, Lingfeng; Fulham, Michael; Feng, Dagan

2014-02-01

In this paper, we address the retrieval of multi-modality medical volumes, which consist of two different imaging modalities, acquired sequentially, from the same scanner. One such example, positron emission tomography and computed tomography (PET-CT), provides physicians with complementary functional and anatomical features as well as spatial relationships and has led to improved cancer diagnosis, localisation, and staging. The challenge of multi-modality volume retrieval for cancer patients lies in representing the complementary geometric and topologic attributes between tumours and organs. These attributes and relationships, which are used for tumour staging and classification, can be formulated as a graph. It has been demonstrated that graph-based methods have high accuracy for retrieval by spatial similarity. However, naïvely representing all relationships on a complete graph obscures the structure of the tumour-anatomy relationships. We propose a new graph structure derived from complete graphs that structurally constrains the edges connected to tumour vertices based upon the spatial proximity of tumours and organs. This enables retrieval on the basis of tumour localisation. We also present a similarity matching algorithm that accounts for different feature sets for graph elements from different imaging modalities. Our method emphasises the relationships between a tumour and related organs, while still modelling patient-specific anatomical variations. Constraining tumours to related anatomical structures improves the discrimination potential of graphs, making it easier to retrieve similar images based on tumour location. We evaluated our retrieval methodology on a dataset of clinical PET-CT volumes. Our results showed that our method enabled the retrieval of multi-modality images using spatial features. Our graph-based retrieval algorithm achieved a higher precision than several other retrieval techniques: gray-level histograms as well as state-of-the-art methods such as visual words using the scale- invariant feature transform (SIFT) and relational matrices representing the spatial arrangements of objects. Copyright © 2013 Elsevier B.V. All rights reserved.

Stimulated Brillouin scattering in highly birefringent multimode tapered chalcogenide photonic crystal fiber for distributed optical sensors (Retraction Notice)

NASA Astrophysics Data System (ADS)

Baili, Amira; Cherif, Rim; Zghal, Mourad

2016-09-01

This paper, originally published on September 15, 2016, was retracted from the SPIE Digital Library on October 5, 2016, due to a high degree of similarity between specific portions of the text of the paper to the following publications: J. Tchahame, J. Beugnot, A. Kudlinski, and T. Sylvestre, "Multimode Brillouin spectrum in a long tapered birefringent photonic crystal fiber," Opt. Lett. 40, 4281-4284 (2015). doi: 10.1364/OL.40.004281 W. W. Ke, X. J. Wang and X. Tang, "Stimulated Brillouin Scattering Model in Multi-Mode Fiber Lasers," in IEEE Journal of Selected Topics in Quantum Electronics, vol. 20, no. 5, pp. 305-314, Sept.-Oct. 2014. doi: 10.1109/JSTQE.2014.2303256.

Stereoscopic Integrated Imaging Goggles for Multimodal Intraoperative Image Guidance

PubMed Central

Mela, Christopher A.; Patterson, Carrie; Thompson, William K.; Papay, Francis; Liu, Yang

2015-01-01

We have developed novel stereoscopic wearable multimodal intraoperative imaging and display systems entitled Integrated Imaging Goggles for guiding surgeries. The prototype systems offer real time stereoscopic fluorescence imaging and color reflectance imaging capacity, along with in vivo handheld microscopy and ultrasound imaging. With the Integrated Imaging Goggle, both wide-field fluorescence imaging and in vivo microscopy are provided. The real time ultrasound images can also be presented in the goggle display. Furthermore, real time goggle-to-goggle stereoscopic video sharing is demonstrated, which can greatly facilitate telemedicine. In this paper, the prototype systems are described, characterized and tested in surgeries in biological tissues ex vivo. We have found that the system can detect fluorescent targets with as low as 60 nM indocyanine green and can resolve structures down to 0.25 mm with large FOV stereoscopic imaging. The system has successfully guided simulated cancer surgeries in chicken. The Integrated Imaging Goggle is novel in 4 aspects: it is (a) the first wearable stereoscopic wide-field intraoperative fluorescence imaging and display system, (b) the first wearable system offering both large FOV and microscopic imaging simultaneously, (c) the first wearable system that offers both ultrasound imaging and fluorescence imaging capacities, and (d) the first demonstration of goggle-to-goggle communication to share stereoscopic views for medical guidance. PMID:26529249

Multi-modal spectroscopic imaging with synchrotron light to study mechanisms of brain disease

NASA Astrophysics Data System (ADS)

Summers, Kelly L.; Fimognari, Nicholas; Hollings, Ashley; Kiernan, Mitchell; Lam, Virginie; Tidy, Rebecca J.; Takechi, Ryu; George, Graham N.; Pickering, Ingrid J.; Mamo, John C.; Harris, Hugh H.; Hackett, Mark J.

2017-04-01

The international health care costs associated with Alzheimer's disease (AD) and dementia have been predicted to reach $2 trillion USD by 2030. As such, there is urgent need to develop new treatments and diagnostic methods to stem an international health crisis. A major limitation to therapy and diagnostic development is the lack of complete understanding about the disease mechanisms. Spectroscopic methods at synchrotron light sources, such as FTIR, XRF, and XAS, offer a "multi-modal imaging platform" to reveal a wealth of important biochemical information in situ within ex vivo tissue sections, to increase our understanding of disease mechanisms.

Multimodal biophotonic workstation for live cell analysis.

PubMed

Esseling, Michael; Kemper, Björn; Antkowiak, Maciej; Stevenson, David J; Chaudet, Lionel; Neil, Mark A A; French, Paul W; von Bally, Gert; Dholakia, Kishan; Denz, Cornelia

2012-01-01

A reliable description and quantification of the complex physiology and reactions of living cells requires a multimodal analysis with various measurement techniques. We have investigated the integration of different techniques into a biophotonic workstation that can provide biological researchers with these capabilities. The combination of a micromanipulation tool with three different imaging principles is accomplished in a single inverted microscope which makes the results from all the techniques directly comparable. Chinese Hamster Ovary (CHO) cells were manipulated by optical tweezers while the feedback was directly analyzed by fluorescence lifetime imaging, digital holographic microscopy and dynamic phase-contrast microscopy. Copyright © 2012 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.

Multimodal optical coherence tomography for in vivo imaging of brain tissue structure and microvascular network at glioblastoma

NASA Astrophysics Data System (ADS)

Yashin, Konstantin S.; Kiseleva, Elena B.; Gubarkova, Ekaterina V.; Matveev, Lev A.; Karabut, Maria M.; Elagin, Vadim V.; Sirotkina, Marina A.; Medyanik, Igor A.; Kravets, L. Y.; Gladkova, Natalia D.

2017-02-01

In the case of infiltrative brain tumors the surgeon faces difficulties in determining their boundaries to achieve total resection. The aim of the investigation was to evaluate the performance of multimodal OCT (MM OCT) for differential diagnostics of normal brain tissue and glioma using an experimental model of glioblastoma. The spectral domain OCT device that was used for the study provides simultaneously two modes: cross-polarization and microangiographic OCT. The comparative analysis of the both OCT modalities images from tumorous and normal brain tissue areas concurrently with histologic correlation shows certain difference between when accordingly to morphological and microvascular tissue features.

Results from the commissioning of a multi-modal endoscope for ultrasound and time of flight PET

DOE Office of Scientific and Technical Information (OSTI.GOV)

Bugalho, Ricardo

2015-07-01

The EndoTOFPET-US collaboration has developed a multi-modal imaging system combining Ultrasound with Time-of-Flight Positron Emission Tomography into an endoscopic imaging device. The objective of the project is to obtain a coincidence time resolution of about 200 ps FWHM and to achieve about 1 mm spatial resolution of the PET system, while integrating all the components in a very compact detector suitable for endoscopic use. This scanner aims to be exploited for diagnostic and surgical oncology, as well as being instrumental in the clinical test of new biomarkers especially targeted for prostate and pancreatic cancer. (authors)

Multimodality imaging to plan and guide transcatheter tricuspid valve interventions.

PubMed

Prihadi, Edgard A; Delgado, Victoria; Bax, Jeroen J

2017-10-01

Tricuspid regurgitation (TR) is a highly prevalent valvular heart disease. The natural history of untreated significant TR portends an unfavorable outcome, but only a minority of patients is currently referred for surgical treatment. Organic TR (caused by primary abnormality of the leaflets) is relatively infrequent whereas secondary or functional TR (caused by dilatation of the tricuspid annulus, right ventricle [RV] and right atrium) is the predominant mechanism. The success of transcatheter therapies for left valvular heart disease over the last decade, has fueled similar development of novel transcatheter devices for the treatment of TR. Currently being tested in several clinical trials, each of these devices requires specific needs to define the procedural suitability. In addition, an accurate evaluation of the complex tricuspid anatomy, RV geometry and their relationship with the surrounding structures is mandatory. Therefore, accurate pre-procedural assessment using multimodality imaging techniques will undoubtedly play a pivotal role in achieving procedural success and safety. This review article provides a comprehensive overview on the etiology and different mechanisms of TR, and highlights the role of multimodality imaging techniques in the assessment of TR severity, RV dysfunction and fulfilment of device-specific selection criteria.

Real-time interactive tractography analysis for multimodal brain visualization tool: MultiXplore

NASA Astrophysics Data System (ADS)

Bakhshmand, Saeed M.; de Ribaupierre, Sandrine; Eagleson, Roy

2017-03-01

Most debilitating neurological disorders can have anatomical origins. Yet unlike other body organs, the anatomy alone cannot easily provide an understanding of brain functionality. In fact, addressing the challenge of linking structural and functional connectivity remains in the frontiers of neuroscience. Aggregating multimodal neuroimaging datasets may be critical for developing theories that span brain functionality, global neuroanatomy and internal microstructures. Functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI) are main such techniques that are employed to investigate the brain under normal and pathological conditions. FMRI records blood oxygenation level of the grey matter (GM), whereas DTI is able to reveal the underlying structure of the white matter (WM). Brain global activity is assumed to be an integration of GM functional hubs and WM neural pathways that serve to connect them. In this study we developed and evaluated a two-phase algorithm. This algorithm is employed in a 3D interactive connectivity visualization framework and helps to accelerate clustering of virtual neural pathways. In this paper, we will detail an algorithm that makes use of an index-based membership array formed for a whole brain tractography file and corresponding parcellated brain atlas. Next, we demonstrate efficiency of the algorithm by measuring required times for extracting a variety of fiber clusters, which are chosen in such a way to resemble all sizes probable output data files that algorithm will generate. The proposed algorithm facilitates real-time visual inspection of neuroimaging data to further the discovery in structure-function relationship of the brain networks.

Use of anomolous thermal imaging effects for multi-mode systems control during crystal growth

NASA Technical Reports Server (NTRS)

Wargo, Michael J.

1989-01-01

Real time image processing techniques, combined with multitasking computational capabilities are used to establish thermal imaging as a multimode sensor for systems control during crystal growth. Whereas certain regions of the high temperature scene are presently unusable for quantitative determination of temperature, the anomalous information thus obtained is found to serve as a potentially low noise source of other important systems control output. Using this approach, the light emission/reflection characteristics of the crystal, meniscus and melt system are used to infer the crystal diameter and a linear regression algorithm is employed to determine the local diameter trend. This data is utilized as input for closed loop control of crystal shape. No performance penalty in thermal imaging speed is paid for this added functionality. Approach to secondary (diameter) sensor design and systems control structure is discussed. Preliminary experimental results are presented.

Advanced multimodality imaging of an anomalous vessel between the ascending aorta and main pulmonary artery in a dog

PubMed Central

Markovic, Lauren E.; Kellihan, Heidi B.; Roldán-Alzate, Alejandro; Drees, Randi; Bjorling, Dale E.; Francois, Chris J.

2014-01-01

A 1-year-old male German shorthaired pointer was referred for evaluation of tachypnea and hemoptysis. A grade VI/VI left basilar continuous murmur was ausculted. Multimodality imaging consisting of thoracic radiographs, transthoracic and transesophageal echocardiography, fluoroscopy-guided selective angiography, computed tomography angiogram (CTA) and magnetic resonance angiogram (MRA), was performed on this patient. The defect included a left-to-right shunting anomalous vessel between the ascending aorta and main pulmonary artery, along with a dissecting aneurysm of the main and right pulmonary artery. An MRA postprocessing technique (PC-VIPR) was used to allow for high resolution angiographic images and further assessment of the patient’s hemodynamics prior to surgical correction. This case report describes the clinical course of a canine patient with a rare form of congenital cardiac disease, and the multiple imaging modalities that were used to aid in diagnosis and treatment. PMID:24485987

Multi-Modal Ultra-Widefield Imaging Features in Waardenburg Syndrome

PubMed Central

Choudhry, Netan; Rao, Rajesh C.

2015-01-01

Background Waardenburg syndrome is characterized by a group of features including; telecanthus, a broad nasal root, synophrys of the eyebrows, piedbaldism, heterochromia irides, and deaf-mutism. Hypopigmentation of the choroid is a unique feature of this condition examined with multi-modal Ultra-Widefield Imaging in this report. Material/Methods Report of a single case. Results Bilateral symmetric choroidal hypopigmentation was observed with hypoautofluorescence in the region of hypopigmentation. Fluorescein angiography revealed a normal vasculature, however a thickened choroid was seen on Enhanced-Depth Imaging Spectral-Domain OCT (EDI SD-OCT). Conclusion(s) Choroidal hypopigmentation is a unique feature of Waardenburg syndrome, which can be visualized with ultra-widefield fundus autofluorescence. The choroid may also be thickened in this condition and its thickness measured with EDI SD-OCT. PMID:26114849

Multimodality Imaging of Myocardial Injury and Remodeling

PubMed Central

Kramer, Christopher M.; Sinusas, Albert J.; Sosnovik, David E.; French, Brent A.; Bengel, Frank M.

2011-01-01

Advances in cardiovascular molecular imaging have come at a rapid pace over the last several years. Multiple approaches have been taken to better understand the structural, molecular, and cellular events that underlie the progression from myocardial injury to myocardial infarction (MI) and, ultimately, to congestive heart failure. Multimodality molecular imaging including SPECT, PET, cardiac MRI, and optical approaches is offering new insights into the pathophysiology of MI and left ventricular remodeling in small-animal models. Targets that are being probed include, among others, angiotensin receptors, matrix metalloproteinases, integrins, apoptosis, macrophages, and sympathetic innervation. It is only a matter of time before these advances are applied in the clinical setting to improve post-MI prognostication and identify appropriate therapies in patients to prevent the onset of congestive heart failure. PMID:20395347

Mapping Epileptic Activity: Sources or Networks for the Clinicians?

PubMed Central

Pittau, Francesca; Mégevand, Pierre; Sheybani, Laurent; Abela, Eugenio; Grouiller, Frédéric; Spinelli, Laurent; Michel, Christoph M.; Seeck, Margitta; Vulliemoz, Serge

2014-01-01

Epileptic seizures of focal origin are classically considered to arise from a focal epileptogenic zone and then spread to other brain regions. This is a key concept for semiological electro-clinical correlations, localization of relevant structural lesions, and selection of patients for epilepsy surgery. Recent development in neuro-imaging and electro-physiology and combinations, thereof, have been validated as contributory tools for focus localization. In parallel, these techniques have revealed that widespread networks of brain regions, rather than a single epileptogenic region, are implicated in focal epileptic activity. Sophisticated multimodal imaging and analysis strategies of brain connectivity patterns have been developed to characterize the spatio-temporal relationships within these networks by combining the strength of both techniques to optimize spatial and temporal resolution with whole-brain coverage and directional connectivity. In this paper, we review the potential clinical contribution of these functional mapping techniques as well as invasive electrophysiology in human beings and animal models for characterizing network connectivity. PMID:25414692

Multimodal medical information retrieval with unsupervised rank fusion.

PubMed

Mourão, André; Martins, Flávio; Magalhães, João

2015-01-01

Modern medical information retrieval systems are paramount to manage the insurmountable quantities of clinical data. These systems empower health care experts in the diagnosis of patients and play an important role in the clinical decision process. However, the ever-growing heterogeneous information generated in medical environments poses several challenges for retrieval systems. We propose a medical information retrieval system with support for multimodal medical case-based retrieval. The system supports medical information discovery by providing multimodal search, through a novel data fusion algorithm, and term suggestions from a medical thesaurus. Our search system compared favorably to other systems in 2013 ImageCLEFMedical. Copyright © 2014 Elsevier Ltd. All rights reserved.

Note: Broadly tunable all-fiber ytterbium laser with 0.05 nm spectral width based on multimode interference filter.

PubMed

Mukhopadhyay, Pranab K; Gupta, Pradeep K; Singh, Amarjeet; Sharma, Sunil K; Bindra, Kushvinder S; Oak, Shrikant M

2014-05-01

A multimode interference filter with narrow transmission bandwidth and large self-imaging wavelength interval is constructed and implemented in an ytterbium doped fiber laser in all-fiber format for broad wavelength tunability as well as narrow spectral width of the output beam. The peak transmission wavelength of the multimode interference filter was tuned with the help of a standard in-fiber polarization controller. With this simple mechanism more than 30 nm (1038 nm-1070 nm) tuning range is demonstrated. The spectral width of the output beam from the laser was measured to be 0.05 nm.

Note: Broadly tunable all-fiber ytterbium laser with 0.05 nm spectral width based on multimode interference filter

NASA Astrophysics Data System (ADS)

Mukhopadhyay, Pranab K.; Gupta, Pradeep K.; Singh, Amarjeet; Sharma, Sunil K.; Bindra, Kushvinder S.; Oak, Shrikant M.

2014-05-01

A multimode interference filter with narrow transmission bandwidth and large self-imaging wavelength interval is constructed and implemented in an ytterbium doped fiber laser in all-fiber format for broad wavelength tunability as well as narrow spectral width of the output beam. The peak transmission wavelength of the multimode interference filter was tuned with the help of a standard in-fiber polarization controller. With this simple mechanism more than 30 nm (1038 nm-1070 nm) tuning range is demonstrated. The spectral width of the output beam from the laser was measured to be 0.05 nm.

Note: Broadly tunable all-fiber ytterbium laser with 0.05 nm spectral width based on multimode interference filter

DOE Office of Scientific and Technical Information (OSTI.GOV)

Mukhopadhyay, Pranab K., E-mail: pkm@rrcat.gov.in; Gupta, Pradeep K.; Singh, Amarjeet

2014-05-15

A multimode interference filter with narrow transmission bandwidth and large self-imaging wavelength interval is constructed and implemented in an ytterbium doped fiber laser in all-fiber format for broad wavelength tunability as well as narrow spectral width of the output beam. The peak transmission wavelength of the multimode interference filter was tuned with the help of a standard in-fiber polarization controller. With this simple mechanism more than 30 nm (1038 nm–1070 nm) tuning range is demonstrated. The spectral width of the output beam from the laser was measured to be 0.05 nm.

Multimodality imaging of reporter gene expression using a novel fusion vector in living cells and animals

DOE Office of Scientific and Technical Information (OSTI.GOV)

Gambhir, Sanjiv; Pritha, Ray

Novel double and triple fusion reporter gene constructs harboring distinct imagable reporter genes are provided, as well as applications for the use of such double and triple fusion constructs in living cells and in living animals using distinct imaging technologies.

Multimodality imaging of reporter gene expression using a novel fusion vector in living cells and animals

DOEpatents

Gambhir, Sanjiv; Pritha, Ray

2015-07-14

Novel double and triple fusion reporter gene constructs harboring distinct imagable reporter genes are provided, as well as applications for the use of such double and triple fusion constructs in living cells and in living animals using distinct imaging technologies.

Multimodal Imaging in Diabetic Macular Edema.

PubMed

Acón, Dhariana; Wu, Lihteh

2018-01-01

Throughout ophthalmic history it has been shown that progress has gone hand in hand with technological breakthroughs. In the past, fluorescein angiography and fundus photographs were the most commonly used imaging modalities in the management of diabetic macular edema (DME). Today, despite the moderate correlation between macular thickness and functional outcomes, spectral domain optical coherence tomography (SD-OCT) has become the DME workhorse in clinical practice. Several SD-OCT biomarkers have been looked at including presence of epiretinal membrane, vitreomacular adhesion, disorganization of the inner retinal layers, central macular thickness, integrity of the ellipsoid layer, and subretinal fluid, among others. Emerging imaging modalities include fundus autofluorescence, macular pigment optical density, fluorescence lifetime imaging ophthalmoscopy, OCT angiography, and adaptive optics. Technological advances in imaging of the posterior segment of the eye have enabled ophthalmologists to develop hypotheses about pathological mechanisms of disease, monitor disease progression, and assess response to treatment. Spectral domain OCT is the most commonly performed imaging modality in the management of DME. However, reliable biomarkers have yet to be identified. Machine learning may provide treatment algorithms based on multimodal imaging. Copyright 2018 Asia-Pacific Academy of Ophthalmology.

A Review of Intravascular Ultrasound–Based Multimodal Intravascular Imaging: The Synergistic Approach to Characterizing Vulnerable Plaques

PubMed Central

Ma, Teng; Zhou, Bill; Hsiai, Tzung K.; Shung, K. Kirk

2015-01-01

Catheter-based intravascular imaging modalities are being developed to visualize pathologies in coronary arteries, such as high-risk vulnerable atherosclerotic plaques known as thin-cap fibroatheroma, to guide therapeutic strategy at preventing heart attacks. Mounting evidences have shown three distinctive histopathological features—the presence of a thin fibrous cap, a lipid-rich necrotic core, and numerous infiltrating macrophages—are key markers of increased vulnerability in atherosclerotic plaques. To visualize these changes, the majority of catheter-based imaging modalities used intravascular ultrasound (IVUS) as the technical foundation and integrated emerging intravascular imaging techniques to enhance the characterization of vulnerable plaques. However, no current imaging technology is the unequivocal “gold standard” for the diagnosis of vulnerable atherosclerotic plaques. Each intravascular imaging technology possesses its own unique features that yield valuable information although encumbered by inherent limitations not seen in other modalities. In this context, the aim of this review is to discuss current scientific innovations, technical challenges, and prospective strategies in the development of IVUS-based multi-modality intravascular imaging systems aimed at assessing atherosclerotic plaque vulnerability. PMID:26400676

Integration of Multi-Modal Biomedical Data to Predict Cancer Grade and Patient Survival.

PubMed

Phan, John H; Hoffman, Ryan; Kothari, Sonal; Wu, Po-Yen; Wang, May D

2016-02-01

The Big Data era in Biomedical research has resulted in large-cohort data repositories such as The Cancer Genome Atlas (TCGA). These repositories routinely contain hundreds of matched patient samples for genomic, proteomic, imaging, and clinical data modalities, enabling holistic and multi-modal integrative analysis of human disease. Using TCGA renal and ovarian cancer data, we conducted a novel investigation of multi-modal data integration by combining histopathological image and RNA-seq data. We compared the performances of two integrative prediction methods: majority vote and stacked generalization. Results indicate that integration of multiple data modalities improves prediction of cancer grade and outcome. Specifically, stacked generalization, a method that integrates multiple data modalities to produce a single prediction result, outperforms both single-data-modality prediction and majority vote. Moreover, stacked generalization reveals the contribution of each data modality (and specific features within each data modality) to the final prediction result and may provide biological insights to explain prediction performance.

Versatile quantitative phase imaging system applied to high-speed, low noise and multimodal imaging (Conference Presentation)

NASA Astrophysics Data System (ADS)

Federici, Antoine; Aknoun, Sherazade; Savatier, Julien; Wattellier, Benoit F.

2017-02-01

Quadriwave lateral shearing interferometry (QWLSI) is a well-established quantitative phase imaging (QPI) technique based on the analysis of interference patterns of four diffraction orders by an optical grating set in front of an array detector [1]. As a QPI modality, this is a non-invasive imaging technique which allow to measure the optical path difference (OPD) of semi-transparent samples. We present a system enabling QWLSI with high-performance sCMOS cameras [2] and apply it to perform high-speed imaging, low noise as well as multimodal imaging. This modified QWLSI system contains a versatile optomechanical device which images the optical grating near the detector plane. Such a device is coupled with any kind of camera by varying its magnification. In this paper, we study the use of a sCMOS Zyla5.5 camera from Andor along with our modified QWLSI system. We will present high-speed live cell imaging, up to 200Hz frame rate, in order to follow intracellular fast motions while measuring the quantitative phase information. The structural and density information extracted from the OPD signal is complementary to the specific and localized fluorescence signal [2]. In addition, QPI detects cells even when the fluorophore is not expressed. This is very useful to follow a protein expression with time. The 10 µm spatial pixel resolution of our modified QWLSI associated to the high sensitivity of the Zyla5.5 enabling to perform high quality fluorescence imaging, we have carried out multimodal imaging revealing fine structures cells, like actin filaments, merged with the morphological information of the phase. References [1]. P. Bon, G. Maucort, B. Wattellier, and S. Monneret, "Quadriwave lateral shearing interferometry for quantitative phase microscopy of living cells," Opt. Express, vol. 17, pp. 13080-13094, 2009. [2] P. Bon, S. Lécart, E. Fort and S. Lévêque-Fort, "Fast label-free cytoskeletal network imaging in living mammalian cells," Biophysical journal, 106(8), pp. 1588-1595, 2014

Multifunctional Fe3O4 @ Au core/shell nanostars: a unique platform for multimode imaging and photothermal therapy of tumors

PubMed Central

Hu, Yong; Wang, Ruizhi; Wang, Shige; Ding, Ling; Li, Jingchao; Luo, Yu; Wang, Xiaolin; Shen, Mingwu; Shi, Xiangyang

2016-01-01

We herein report the development of multifunctional folic acid (FA)-targeted Fe3O4 @ Au nanostars (NSs) for targeted multi-mode magnetic resonance (MR)/computed tomography (CT)/photoacoustic (PA) imaging and photothermal therapy (PTT) of tumors. In this present work, citric acid-stabilized Fe3O4/Ag composite nanoparticles prepared by a mild reduction route were utilized as seeds and exposed to the Au growth solution to induce the formation of Fe3O4 @ Au core/shell NSs. Followed by successive decoration of thiolated polyethyleneimine (PEI-SH), FA via a polyethylene glycol spacer, and acetylation of the residual PEI amines, multifunctional Fe3O4 @ Au NSs were formed. The designed multifunctional NSs possess excellent colloidal stability, good cytocompatibility in a given concentration range, and specific recognition to cancer cells overexpressing FA receptors. Due to co-existence of Fe3O4 core and star-shaped Au shell, the NSs can be used for MR and CT imaging of tumors, respectively. Likewise, the near infrared plasmonic absorption feature also enables the NSs to be used for PA imaging and PTT of tumors. Our study clearly demonstrates a unique theranostic nanoplatform that can be used for high performance multi-mode imaging-guided PTT of tumors, which may be extendable for theranostics of different diseases in translational medicine. PMID:27325015

A practical salient region feature based 3D multi-modality registration method for medical images

NASA Astrophysics Data System (ADS)

Hahn, Dieter A.; Wolz, Gabriele; Sun, Yiyong; Hornegger, Joachim; Sauer, Frank; Kuwert, Torsten; Xu, Chenyang

2006-03-01

We present a novel representation of 3D salient region features and its integration into a hybrid rigid-body registration framework. We adopt scale, translation and rotation invariance properties of those intrinsic 3D features to estimate a transform between underlying mono- or multi-modal 3D medical images. Our method combines advantageous aspects of both feature- and intensity-based approaches and consists of three steps: an automatic extraction of a set of 3D salient region features on each image, a robust estimation of correspondences and their sub-pixel accurate refinement with outliers elimination. We propose a region-growing based approach for the extraction of 3D salient region features, a solution to the problem of feature clustering and a reduction of the correspondence search space complexity. Results of the developed algorithm are presented for both mono- and multi-modal intra-patient 3D image pairs (CT, PET and SPECT) that have been acquired for change detection, tumor localization, and time based intra-person studies. The accuracy of the method is clinically evaluated by a medical expert with an approach that measures the distance between a set of selected corresponding points consisting of both anatomical and functional structures or lesion sites. This demonstrates the robustness of the proposed method to image overlap, missing information and artefacts. We conclude by discussing potential medical applications and possibilities for integration into a non-rigid registration framework.

Multimodal imaging of the disease progression of birdshot chorioretinopathy.

PubMed

Teussink, Michel M; Huis In Het Veld, Paulien I; de Vries, Lieuwe A M; Hoyng, Carel B; Klevering, B Jeroen; Theelen, Thomas

2016-12-01

To study outer retinal deterioration in relation to clinical disease activity in patients with birdshot chorioretinopathy using fundus autofluorescence and spectral-domain optical coherence tomography (OCT). A single-centre retrospective cohort study was carried out on 42 eyes of 21 patients with birdshot disease, using a multimodal imaging approach including fundus autofluorescence, OCT, fluorescein angiography and indocyanine green angiography in combination with a patient chart review. The patients' overall clinical activity of retinal vasculitis during the follow-up period was determined by periods of clinical activity as indicated by fluorescein angiography and associated treatment decisions. Image analysis was performed to examine the spatial correspondence between autofluorescence changes and disruption of the photoreceptor inner segment ellipsoid zone on OCT. Three common types of outer retinal lesions were observed in fovea-centred images of 43% of patients: circular patches of chorioretinal atrophy, ellipsoid zone disruption on OCT, and outer retinal atrophy on autofluorescence and OCT. There was good spatial correspondence between ellipsoid zone disruption and areas of diffuse hyper-autofluorescence outside the fovea. Interestingly, the ellipsoid zone disruption recovered in four out of seven patients upon intensified therapeutic immunosuppression. Most patients only developed peripapillary atrophy and occasional perivascular hypo-autofluorescence. A multimodal imaging approach with autofluorescence imaging and OCT may help to detect ellipsoid zone disruption in the central retina of patients with birdshot disease. Our results suggest that ellipsoid zone disruption may be related to both the activity and duration of retinal vasculitis, and could help to determine therapeutic success in birdshot disease. © 2016 Acta Ophthalmologica Scandinavica Foundation. Published by John Wiley & Sons Ltd.

Multimodal Chemical Imaging of Amyloid Plaque Polymorphism Reveals Aβ Aggregation Dependent Anionic Lipid Accumulations and Metabolism.

PubMed

Michno, Wojciech; Kaya, Ibrahim; Nyström, Sofie; Guerard, Laurent; Nilsson, K Peter R; Hammarström, Per; Blennow, Kaj; Zetterberg, Henrik; Hanrieder, Jörg

2018-06-01

Amyloid plaque formation constitutes one of the main pathological hallmark of Alzheimer's disease (AD) and is suggested to be a critical factor driving disease pathogenesis. Interestingly, in patients that display amyloid pathology but remain cognitively normal, Aβ deposits are predominantly of diffuse morphology suggesting that cored plaque formation is primarily associated with cognitive deterioration and AD pathogenesis. Little is known about the molecular mechanism responsible for conversion of monomeric Aβ into neurotoxic aggregates and the predominantly cored deposits observed in AD. The structural diversity among Aβ plaques, including cored/compact- and diffuse, may be linked to their distinct Aβ profile and other chemical species including neuronal lipids. We developed a novel, chemical imaging paradigm combining matrix assisted laser desorption/ionization imaging mass spectrometry (MALDI IMS) and fluorescent amyloid staining. This multimodal imaging approach was used to probe the lipid chemistry associated with structural plaque heterogeneity in transgenic AD mice (tgAPPSwe) and was correlated to Aβ profiles determined by subsequent laser microdissection and immunoprecipitation-mass spectrometry. Multivariate image analysis revealed an inverse localization of ceramides and their matching metabolites to diffuse and cored structures within single plaques, respectively. Moreover, phosphatidylinositols implicated in AD pathogenesis, were found to localise to the diffuse Aβ structures and correlate with Aβ1-42. Further, lysophospholipids implicated in neuroinflammation were increased in all Aβ deposits. The results support previous clinical findings on the importance of lipid disturbances in AD pathophysiology and associated sphingolipid processing. These data highlight the potential of multimodal imaging as a powerful technology to probe neuropathological mechanisms.

A digital 3D atlas of the marmoset brain based on multi-modal MRI.

PubMed

Liu, Cirong; Ye, Frank Q; Yen, Cecil Chern-Chyi; Newman, John D; Glen, Daniel; Leopold, David A; Silva, Afonso C

2018-04-01

The common marmoset (Callithrix jacchus) is a New-World monkey of growing interest in neuroscience. Magnetic resonance imaging (MRI) is an essential tool to unveil the anatomical and functional organization of the marmoset brain. To facilitate identification of regions of interest, it is desirable to register MR images to an atlas of the brain. However, currently available atlases of the marmoset brain are mainly based on 2D histological data, which are difficult to apply to 3D imaging techniques. Here, we constructed a 3D digital atlas based on high-resolution ex-vivo MRI images, including magnetization transfer ratio (a T1-like contrast), T2w images, and multi-shell diffusion MRI. Based on the multi-modal MRI images, we manually delineated 54 cortical areas and 16 subcortical regions on one hemisphere of the brain (the core version). The 54 cortical areas were merged into 13 larger cortical regions according to their locations to yield a coarse version of the atlas, and also parcellated into 106 sub-regions using a connectivity-based parcellation method to produce a refined atlas. Finally, we compared the new atlas set with existing histology atlases and demonstrated its applications in connectome studies, and in resting state and stimulus-based fMRI. The atlas set has been integrated into the widely-distributed neuroimaging data analysis software AFNI and SUMA, providing a readily usable multi-modal template space with multi-level anatomical labels (including labels from the Paxinos atlas) that can facilitate various neuroimaging studies of marmosets. Published by Elsevier Inc.

Multimodal MSI in Conjunction with Broad Coverage Spatially Resolved MS 2 Increases Confidence in Both Molecular Identification and Localization

DOE Office of Scientific and Technical Information (OSTI.GOV)

Veličković, Dušan; Chu, Rosalie K.; Carrell, Alyssa A.

One critical aspect of mass spectrometry imaging (MSI) is the need to confidently identify detected analytes. While orthogonal tandem MS (e.g., LC-MS 2) experiments from sample extracts can assist in annotating ions, the spatial information about these molecules is lost. Accordingly, this could cause mislead conclusions, especially in cases where isobaric species exhibit different distributions within a sample. In this Technical Note, we employed a multimodal imaging approach, using matrix assisted laser desorption/ionization (MALDI)-MSI and liquid extraction surface analysis (LESA)-MS 2I, to confidently annotate and One critical aspect of mass spectrometry imaging (MSI) is the need to confidently identify detectedmore » analytes. While orthogonal tandem MS (e.g., LC-MS2) experiments from sample extracts can assist in annotating ions, the spatial information about these molecules is lost. Accordingly, this could cause mislead conclusions, especially in cases where isobaric species exhibit different distributions within a sample. In this Technical Note, we employed a multimodal imaging approach, using matrix assisted laser desorption/ionization (MALDI)-MSI and liquid extraction surface analysis (LESA)-MS 2I, to confidently annotate and localize a broad range of metabolites involved in a tripartite symbiosis system of moss, cyanobacteria, and fungus. We found that the combination of these two imaging modalities generated very congruent ion images, providing the link between highly accurate structural information onfered by LESA and high spatial resolution attainable by MALDI. These results demonstrate how this combined methodology could be very useful in differentiating metabolite routes in complex systems.« less

IMAGING WITH MULTIMODAL ADAPTIVE-OPTICS OPTICAL COHERENCE TOMOGRAPHY IN MULTIPLE EVANESCENT WHITE DOT SYNDROME: THE STRUCTURE AND FUNCTIONAL RELATIONSHIP.

PubMed

Labriola, Leanne T; Legarreta, Andrew D; Legarreta, John E; Nadler, Zach; Gallagher, Denise; Hammer, Daniel X; Ferguson, R Daniel; Iftimia, Nicusor; Wollstein, Gadi; Schuman, Joel S

2016-01-01

To elucidate the location of pathological changes in multiple evanescent white dot syndrome (MEWDS) with the use of multimodal adaptive optics (AO) imaging. A 5-year observational case study of a 24-year-old female with recurrent MEWDS. Full examination included history, Snellen chart visual acuity, pupil assessment, intraocular pressures, slit lamp evaluation, dilated fundoscopic exam, imaging with Fourier-domain optical coherence tomography (FD-OCT), blue-light fundus autofluorescence (FAF), fundus photography, fluorescein angiography, and adaptive-optics optical coherence tomography. Three distinct acute episodes of MEWDS occurred during the period of follow-up. Fourier-domain optical coherence tomography and adaptive-optics imaging showed disturbance in the photoreceptor outer segments (PR OS) in the posterior pole with each flare. The degree of disturbance at the photoreceptor level corresponded to size and extent of the visual field changes. All findings were transient with delineation of the photoreceptor recovery from the outer edges of the lesion inward. Hyperautofluorescence was seen during acute flares. Increase in choroidal thickness did occur with each active flare but resolved. Although changes in the choroid and RPE can be observed in MEWDS, Fourier-domain optical coherence tomography, and multimodal adaptive optics imaging localized the visually significant changes seen in this disease at the level of the photoreceptors. These transient retinal changes specifically occur at the level of the inner segment ellipsoid and OS/RPE line. En face optical coherence tomography imaging provides a detailed, yet noninvasive method for following the convalescence of MEWDS and provides insight into the structural and functional relationship of this transient inflammatory retinal disease.

Multimodal registration via spatial-context mutual information.

PubMed

Yi, Zhao; Soatto, Stefano

2011-01-01

We propose a method to efficiently compute mutual information between high-dimensional distributions of image patches. This in turn is used to perform accurate registration of images captured under different modalities, while exploiting their local structure otherwise missed in traditional mutual information definition. We achieve this by organizing the space of image patches into orbits under the action of Euclidean transformations of the image plane, and estimating the modes of a distribution in such an orbit space using affinity propagation. This way, large collections of patches that are equivalent up to translations and rotations are mapped to the same representative, or "dictionary element". We then show analytically that computing mutual information for a joint distribution in this space reduces to computing mutual information between the (scalar) label maps, and between the transformations mapping each patch into its closest dictionary element. We show that our approach improves registration performance compared with the state of the art in multimodal registration, using both synthetic and real images with quantitative ground truth.

Multimodality 3D Superposition and Automated Whole Brain Tractography: Comprehensive Printing of the Functional Brain

PubMed Central

Brimley, Cameron J; Sublett, Jesna Mathew; Stefanowicz, Edward; Flora, Sarah; Mongelluzzo, Gino; Schirmer, Clemens M

2017-01-01

Whole brain tractography using diffusion tensor imaging (DTI) sequences can be used to map cerebral connectivity; however, this can be time-consuming due to the manual component of image manipulation required, calling for the need for a standardized, automated, and accurate fiber tracking protocol with automatic whole brain tractography (AWBT). Interpreting conventional two-dimensional (2D) images, such as computed tomography (CT) and magnetic resonance imaging (MRI), as an intraoperative three-dimensional (3D) environment is a difficult task with recognized inter-operator variability. Three-dimensional printing in neurosurgery has gained significant traction in the past decade, and as software, equipment, and practices become more refined, trainee education, surgical skills, research endeavors, innovation, patient education, and outcomes via valued care is projected to improve. We describe a novel multimodality 3D superposition (MMTS) technique, which fuses multiple imaging sequences alongside cerebral tractography into one patient-specific 3D printed model. Inferences on cost and improved outcomes fueled by encouraging patient engagement are explored. PMID:29201580

Optical and nuclear imaging of glioblastoma with phosphatidylserine-targeted nanovesicles.

PubMed

Blanco, Víctor M; Chu, Zhengtao; LaSance, Kathleen; Gray, Brian D; Pak, Koon Yan; Rider, Therese; Greis, Kenneth D; Qi, Xiaoyang

2016-05-31

Multimodal tumor imaging with targeted nanoparticles potentially offers both enhanced specificity and sensitivity, leading to more precise cancer diagnosis and monitoring. We describe the synthesis and characterization of phenol-substituted, lipophilic orange and far-red fluorescent dyes and a simple radioiodination procedure to generate a dual (optical and nuclear) imaging probe. MALDI-ToF analyses revealed high iodination efficiency of the lipophilic reporters, achieved by electrophilic aromatic substitution using the chloramide 1,3,4,6-tetrachloro-3α,6α-diphenyl glycoluril (Iodogen) as the oxidizing agent in an organic/aqueous co-solvent mixture. Upon conjugation of iodine-127 or iodine-124-labeled reporters to tumor-targeting SapC-DOPS nanovesicles, optical (fluorescent) and PET imaging was performed in mice bearing intracranial glioblastomas. In addition, tumor vs non-tumor (normal brain) uptake was compared using iodine-125. These data provide proof-of-principle for the potential value of SapC-DOPS for multimodal imaging of glioblastoma, the most aggressive primary brain tumor.

Nanoparticles speckled by ready-to-conjugate lanthanide complexes for multimodal imaging

NASA Astrophysics Data System (ADS)

Biju, Vasudevanpillai; Hamada, Morihiko; Ono, Kenji; Sugino, Sakiko; Ohnishi, Takashi; Shibu, Edakkattuparambil Sidharth; Yamamura, Shohei; Sawada, Makoto; Nakanishi, Shunsuke; Shigeri, Yasushi; Wakida, Shin-Ichi

2015-09-01

Multimodal and multifunctional contrast agents receive enormous attention in the biomedical imaging field. Such contrast agents are routinely prepared by the incorporation of organic molecules and inorganic nanoparticles (NPs) into host materials such as gold NPs, silica NPs, polymer NPs, and liposomes. Despite their non-cytotoxic nature, the large size of these NPs limits the in vivo distribution and clearance and inflames complex pharmacokinetics, which hinder the regulatory approval for clinical applications. Herein, we report a unique method that combines magnetic resonance imaging (MRI) and fluorescence imaging modalities together in nanoscale entities by the simple, direct and stable conjugation of novel biotinylated coordination complexes of gadolinium(iii) to CdSe/ZnS quantum dots (QD) and terbium(iii) to super paramagnetic iron oxide NPs (SPION) but without any host material. Subsequently, we evaluate the potentials of such lanthanide-speckled fluorescent-magnetic NPs for bioimaging at single-molecule, cell and in vivo levels. The simple preparation and small size make such fluorescent-magnetic NPs promising contrast agents for biomedical imaging.

Highlights lecture EANM 2016: "Embracing molecular imaging and multi-modal imaging: a smart move for nuclear medicine towards personalized medicine".

PubMed

Aboagye, Eric O; Kraeber-Bodéré, Françoise

2017-08-01

The 2016 EANM Congress took place in Barcelona, Spain, from 15 to 19 October under the leadership of Prof. Wim Oyen, chair of the EANM Scientific Committee. With more than 6,000 participants, this congress was the most important European event in nuclear medicine, bringing together a multidisciplinary community involved in the different fields of nuclear medicine. There were over 600 oral and 1,200 poster or e-Poster presentations with an overwhelming focus on development and application of imaging for personalized care, which is timely for the community. Beyond FDG PET, major highlights included progress in the use of PSMA and SSTR receptor-targeted radiopharmaceuticals and associated theranostics in oncology. Innovations in radiopharmaceuticals for imaging pathologies of the brain and cardiovascular system, as well as infection and inflammation, were also highlighted. In the areas of physics and instrumentation, multimodality imaging and radiomics were highlighted as promising areas of research.

Separating Bulk and Surface Contributions to Electronic Excited-State Processes in Hybrid Mixed Perovskite Thin Films via Multimodal All-Optical Imaging.

PubMed

Simpson, Mary Jane; Doughty, Benjamin; Das, Sanjib; Xiao, Kai; Ma, Ying-Zhong

2017-07-20

A comprehensive understanding of electronic excited-state phenomena underlying the impressive performance of solution-processed hybrid halide perovskite solar cells requires access to both spatially resolved electronic processes and corresponding sample morphological characteristics. Here, we demonstrate an all-optical multimodal imaging approach that enables us to obtain both electronic excited-state and morphological information on a single optical microscope platform with simultaneous high temporal and spatial resolution. Specifically, images were acquired for the same region of interest in thin films of chloride containing mixed lead halide perovskites (CH 3 NH 3 PbI 3-x Cl x ) using femtosecond transient absorption, time-integrated photoluminescence, confocal reflectance, and transmission microscopies. Comprehensive image analysis revealed the presence of surface- and bulk-dominated contributions to the various images, which describe either spatially dependent electronic excited-state properties or morphological variations across the probed region of the thin films. These results show that PL probes effectively the species near or at the film surface.

Multimodality 3D Superposition and Automated Whole Brain Tractography: Comprehensive Printing of the Functional Brain.

PubMed

Konakondla, Sanjay; Brimley, Cameron J; Sublett, Jesna Mathew; Stefanowicz, Edward; Flora, Sarah; Mongelluzzo, Gino; Schirmer, Clemens M

2017-09-29

Whole brain tractography using diffusion tensor imaging (DTI) sequences can be used to map cerebral connectivity; however, this can be time-consuming due to the manual component of image manipulation required, calling for the need for a standardized, automated, and accurate fiber tracking protocol with automatic whole brain tractography (AWBT). Interpreting conventional two-dimensional (2D) images, such as computed tomography (CT) and magnetic resonance imaging (MRI), as an intraoperative three-dimensional (3D) environment is a difficult task with recognized inter-operator variability. Three-dimensional printing in neurosurgery has gained significant traction in the past decade, and as software, equipment, and practices become more refined, trainee education, surgical skills, research endeavors, innovation, patient education, and outcomes via valued care is projected to improve. We describe a novel multimodality 3D superposition (MMTS) technique, which fuses multiple imaging sequences alongside cerebral tractography into one patient-specific 3D printed model. Inferences on cost and improved outcomes fueled by encouraging patient engagement are explored.

Multimodal imaging of language reorganization in patients with left temporal lobe epilepsy.

PubMed

Chang, Yu-Hsuan A; Kemmotsu, Nobuko; Leyden, Kelly M; Kucukboyaci, N Erkut; Iragui, Vicente J; Tecoma, Evelyn S; Kansal, Leena; Norman, Marc A; Compton, Rachelle; Ehrlich, Tobin J; Uttarwar, Vedang S; Reyes, Anny; Paul, Brianna M; McDonald, Carrie R

2017-07-01

This study explored the relationships among multimodal imaging, clinical features, and language impairment in patients with left temporal lobe epilepsy (LTLE). Fourteen patients with LTLE and 26 controls underwent structural MRI, functional MRI, diffusion tensor imaging, and neuropsychological language tasks. Laterality indices were calculated for each imaging modality and a principal component (PC) was derived from language measures. Correlations were performed among imaging measures, as well as to the language PC. In controls, better language performance was associated with stronger left-lateralized temporo-parietal and temporo-occipital activations. In LTLE, better language performance was associated with stronger right-lateralized inferior frontal, temporo-parietal, and temporo-occipital activations. These right-lateralized activations in LTLE were associated with right-lateralized arcuate fasciculus fractional anisotropy. These data suggest that interhemispheric language reorganization in LTLE is associated with alterations to perisylvian white matter. These concurrent structural and functional shifts from left to right may help to mitigate language impairment in LTLE. Copyright © 2017 Elsevier Inc. All rights reserved.

Robust multi-site MR data processing: iterative optimization of bias correction, tissue classification, and registration.

PubMed

Young Kim, Eun; Johnson, Hans J

2013-01-01

A robust multi-modal tool, for automated registration, bias correction, and tissue classification, has been implemented for large-scale heterogeneous multi-site longitudinal MR data analysis. This work focused on improving the an iterative optimization framework between bias-correction, registration, and tissue classification inspired from previous work. The primary contributions are robustness improvements from incorporation of following four elements: (1) utilize multi-modal and repeated scans, (2) incorporate high-deformable registration, (3) use extended set of tissue definitions, and (4) use of multi-modal aware intensity-context priors. The benefits of these enhancements were investigated by a series of experiments with both simulated brain data set (BrainWeb) and by applying to highly-heterogeneous data from a 32 site imaging study with quality assessments through the expert visual inspection. The implementation of this tool is tailored for, but not limited to, large-scale data processing with great data variation with a flexible interface. In this paper, we describe enhancements to a joint registration, bias correction, and the tissue classification, that improve the generalizability and robustness for processing multi-modal longitudinal MR scans collected at multi-sites. The tool was evaluated by using both simulated and simulated and human subject MRI images. With these enhancements, the results showed improved robustness for large-scale heterogeneous MRI processing.

Stereotactic multibeam radiation therapy system in a PACS environment

NASA Astrophysics Data System (ADS)

Fresne, Francoise; Le Gall, G.; Barillot, Christian; Gibaud, Bernard; Manens, Jean-Pierre; Toumoulin, Christine; Lemoine, Didier; Chenal, C.; Scarabin, Jean-Marie

1991-05-01

A Multibeam radiation therapy treatment is a non-invasive technique devoted to treat a lesion within the cerebral medium by focusing photon-beams on the same target from a high number of entrance points. We present here a computer assisted dosimetric planning procedure which includes: (1) an analysis module to define the target volume by using 2D and 3D displays, (2) a planing module to issue a treatment strategy including the dosimetric simulations and (3) a treatment module setting up the parameters to order the robotized treatment system (i.e. chair- framework, radiation unit machine). Another important feature of this system is its connection to the PACS system SIRENE settled in the University hospital of Rennes which makes possible the archiving and the communication of the multimodal images (CT, MRI, Angiography) used by this application. The corporate use of stereotactic methods and the multimodality imagery ensures spatial coherence and makes the target definition and the cognition of the structures environment more accurate. The dosimetric planning suited to the spatial reference (i.e. the stereotactic frame) guarantees an optimal distribution of the dose computed by an original 3D volumetric algorithm. The robotic approach of the treatment stage has consisted to design a computer driven chair-framework cluster to position the target volume at the radiation unit isocenter.

A multimodal 3D framework for fire characteristics estimation

NASA Astrophysics Data System (ADS)

Toulouse, T.; Rossi, L.; Akhloufi, M. A.; Pieri, A.; Maldague, X.

2018-02-01

In the last decade we have witnessed an increasing interest in using computer vision and image processing in forest fire research. Image processing techniques have been successfully used in different fire analysis areas such as early detection, monitoring, modeling and fire front characteristics estimation. While the majority of the work deals with the use of 2D visible spectrum images, recent work has introduced the use of 3D vision in this field. This work proposes a new multimodal vision framework permitting the extraction of the three-dimensional geometrical characteristics of fires captured by multiple 3D vision systems. The 3D system is a multispectral stereo system operating in both the visible and near-infrared (NIR) spectral bands. The framework supports the use of multiple stereo pairs positioned so as to capture complementary views of the fire front during its propagation. Multimodal registration is conducted using the captured views in order to build a complete 3D model of the fire front. The registration process is achieved using multisensory fusion based on visual data (2D and NIR images), GPS positions and IMU inertial data. Experiments were conducted outdoors in order to show the performance of the proposed framework. The obtained results are promising and show the potential of using the proposed framework in operational scenarios for wildland fire research and as a decision management system in fighting.

Tunable and noncytotoxic PET/SPECT-MRI multimodality imaging probes using colloidally stable ligand-free superparamagnetic iron oxide nanoparticles

PubMed Central

Pham, TH Nguyen; Lengkeek, Nigel A; Greguric, Ivan; Kim, Byung J; Pellegrini, Paul A; Bickley, Stephanie A; Tanudji, Marcel R; Jones, Stephen K; Hawkett, Brian S; Pham, Binh TT

2017-01-01

Physiologically stable multimodality imaging probes for positron emission tomography/single-photon emission computed tomography (PET/SPECT)-magnetic resonance imaging (MRI) were synthesized using the superparamagnetic maghemite iron oxide (γ-Fe2O3) nanoparticles (SPIONs). The SPIONs were sterically stabilized with a finely tuned mixture of diblock copolymers with either methoxypolyethylene glycol (MPEG) or primary amine NH2 end groups. The radioisotope for PET or SPECT imaging was incorporated with the SPIONs at high temperature. 57Co2+ ions with a long half-life of 270.9 days were used as a model for the radiotracer to study the kinetics of radiolabeling, characterization, and the stability of the radiolabeled SPIONs. Radioactive 67Ga3+ and Cu2+-labeled SPIONs were also produced successfully using the optimized conditions from the 57Co2+-labeling process. No free radioisotopes were detected in the aqueous phase for the radiolabeled SPIONs 1 week after dispersion in phosphate-buffered saline (PBS). All labeled SPIONs were not only well dispersed and stable under physiological conditions but also noncytotoxic in vitro. The ability to design and produce physiologically stable radiolabeled magnetic nanoparticles with a finely controlled number of functionalizable end groups on the SPIONs enables the generation of a desirable and biologically compatible multimodality PET/SPECT-MRI agent on a single T2 contrast MRI probe. PMID:28184160

A framework for biomedical figure segmentation towards image-based document retrieval

PubMed Central

2013-01-01

The figures included in many of the biomedical publications play an important role in understanding the biological experiments and facts described within. Recent studies have shown that it is possible to integrate the information that is extracted from figures in classical document classification and retrieval tasks in order to improve their accuracy. One important observation about the figures included in biomedical publications is that they are often composed of multiple subfigures or panels, each describing different methodologies or results. The use of these multimodal figures is a common practice in bioscience, as experimental results are graphically validated via multiple methodologies or procedures. Thus, for a better use of multimodal figures in document classification or retrieval tasks, as well as for providing the evidence source for derived assertions, it is important to automatically segment multimodal figures into subfigures and panels. This is a challenging task, however, as different panels can contain similar objects (i.e., barcharts and linecharts) with multiple layouts. Also, certain types of biomedical figures are text-heavy (e.g., DNA sequences and protein sequences images) and they differ from traditional images. As a result, classical image segmentation techniques based on low-level image features, such as edges or color, are not directly applicable to robustly partition multimodal figures into single modal panels. In this paper, we describe a robust solution for automatically identifying and segmenting unimodal panels from a multimodal figure. Our framework starts by robustly harvesting figure-caption pairs from biomedical articles. We base our approach on the observation that the document layout can be used to identify encoded figures and figure boundaries within PDF files. Taking into consideration the document layout allows us to correctly extract figures from the PDF document and associate their corresponding caption. We combine pixel-level representations of the extracted images with information gathered from their corresponding captions to estimate the number of panels in the figure. Thus, our approach simultaneously identifies the number of panels and the layout of figures. In order to evaluate the approach described here, we applied our system on documents containing protein-protein interactions (PPIs) and compared the results against a gold standard that was annotated by biologists. Experimental results showed that our automatic figure segmentation approach surpasses pure caption-based and image-based approaches, achieving a 96.64% accuracy. To allow for efficient retrieval of information, as well as to provide the basis for integration into document classification and retrieval systems among other, we further developed a web-based interface that lets users easily retrieve panels containing the terms specified in the user queries. PMID:24565394

Heterogeneous Optimization Framework: Reproducible Preprocessing of Multi-Spectral Clinical MRI for Neuro-Oncology Imaging Research.

PubMed

Milchenko, Mikhail; Snyder, Abraham Z; LaMontagne, Pamela; Shimony, Joshua S; Benzinger, Tammie L; Fouke, Sarah Jost; Marcus, Daniel S

2016-07-01

Neuroimaging research often relies on clinically acquired magnetic resonance imaging (MRI) datasets that can originate from multiple institutions. Such datasets are characterized by high heterogeneity of modalities and variability of sequence parameters. This heterogeneity complicates the automation of image processing tasks such as spatial co-registration and physiological or functional image analysis. Given this heterogeneity, conventional processing workflows developed for research purposes are not optimal for clinical data. In this work, we describe an approach called Heterogeneous Optimization Framework (HOF) for developing image analysis pipelines that can handle the high degree of clinical data non-uniformity. HOF provides a set of guidelines for configuration, algorithm development, deployment, interpretation of results and quality control for such pipelines. At each step, we illustrate the HOF approach using the implementation of an automated pipeline for Multimodal Glioma Analysis (MGA) as an example. The MGA pipeline computes tissue diffusion characteristics of diffusion tensor imaging (DTI) acquisitions, hemodynamic characteristics using a perfusion model of susceptibility contrast (DSC) MRI, and spatial cross-modal co-registration of available anatomical, physiological and derived patient images. Developing MGA within HOF enabled the processing of neuro-oncology MR imaging studies to be fully automated. MGA has been successfully used to analyze over 160 clinical tumor studies to date within several research projects. Introduction of the MGA pipeline improved image processing throughput and, most importantly, effectively produced co-registered datasets that were suitable for advanced analysis despite high heterogeneity in acquisition protocols.

Development of a small prototype for a proof-of-concept of OpenPET imaging

NASA Astrophysics Data System (ADS)

Yamaya, Taiga; Yoshida, Eiji; Inaniwa, Taku; Sato, Shinji; Nakajima, Yasunori; Wakizaka, Hidekatsu; Kokuryo, Daisuke; Tsuji, Atsushi; Mitsuhashi, Takayuki; Kawai, Hideyuki; Tashima, Hideaki; Nishikido, Fumihiko; Inadama, Naoko; Murayama, Hideo; Haneishi, Hideaki; Suga, Mikio; Kinouchi, Shoko

2011-02-01

The OpenPET geometry is our new idea to visualize a physically opened space between two detector rings. In this paper, we developed the first small prototype to show a proof-of-concept of OpenPET imaging. Two detector rings of 110 mm diameter and 42 mm axial length were placed with a gap of 42 mm. The basic imaging performance was confirmed through phantom studies; the open imaging was realized at the cost of slight loss of axial resolution and 24% loss of sensitivity. For a proof-of-concept of PET image-guided radiation therapy, we carried out the in-beam tests with 11C radioactive beam irradiation in the heavy ion medical accelerator in Chiba to visualize in situ distribution of primary particles stopped in a phantom. We showed that PET images corresponding to dose distribution were obtained. For an initial proof-of-concept of real-time multimodal imaging, we measured a tumor-inoculated mouse with 18F-FDG, and an optical image of the mouse body surface was taken during the PET measurement by inserting a digital camera in the ring gap. We confirmed that the tumor in the gap was clearly visualized. The result also showed the extension effect of an axial field-of-view (FOV); a large axial FOV of 126 mm was obtained with the detectors that originally covered only an 84 mm axial FOV. In conclusion, our initial imaging studies showed promising performance of the OpenPET.

Multi-focus beam shaping of high power multimode lasers

NASA Astrophysics Data System (ADS)

Laskin, Alexander; Volpp, Joerg; Laskin, Vadim; Ostrun, Aleksei

2017-08-01

Beam shaping of powerful multimode fiber lasers, fiber-coupled solid-state and diode lasers is of great importance for improvements of industrial laser applications. Welding, cladding with millimetre scale working spots benefit from "inverseGauss" intensity profiles; performance of thick metal sheet cutting, deep penetration welding can be enhanced when distributing the laser energy along the optical axis as more efficient usage of laser energy, higher edge quality and reduction of the heat affected zone can be achieved. Building of beam shaping optics for multimode lasers encounters physical limitations due to the low beam spatial coherence of multimode fiber-coupled lasers resulting in big Beam Parameter Products (BPP) or M² values. The laser radiation emerging from a multimode fiber presents a mixture of wavefronts. The fiber end can be considered as a light source which optical properties are intermediate between a Lambertian source and a single mode laser beam. Imaging of the fiber end, using a collimator and a focusing objective, is a robust and widely used beam delivery approach. Beam shaping solutions are suggested in form of optics combining fiber end imaging and geometrical separation of focused spots either perpendicular to or along the optical axis. Thus, energy of high power lasers is distributed among multiple foci. In order to provide reliable operation with multi-kW lasers and avoid damages the optics are designed as refractive elements with smooth optical surfaces. The paper presents descriptions of multi-focus optics as well as examples of intensity profile measurements of beam caustics and application results.

Fluorescent magnetic hybrid nanoprobe for multimodal bioimaging

PubMed Central

Bright, Vanessa

2011-01-01

A fluorescent magnetic hybrid imaging nanoprobe (HINP) was fabricated by conjugation of superparamagnetic Fe3O4 nanoparticles and visible light-emitting (~600 nm) fluorescent CdTe/CdS quantum dots (QDs). The assembly strategy used the covalent linking of the oxidized dextran shell of magnetic particles to the glutathione ligands of QDs. Synthesized HINP formed stable water-soluble colloidal dispersions. The structure and properties of the particles were characterized by transmission electron and atomic force microscopy, energy dispersive X-ray analysis and inductively coupled plasma optical emission spectroscopy, dynamic light scattering analysis, optical absorption and photoluminescence spectroscopy, and fluorescent imaging. The luminescence imaging region of the nanoprobe was extended to the near-infrared (NIR) (~800 nm) by conjugation of superparamagnetic nanoparticles with synthesized CdHgTe/CdS QDs. Cadmium, mercury based QDs in HINP can be easily replaced by novel water soluble glutathione stabilized AgInS2/ZnS QDs to present a new class of cadmium-free multimodal imaging agents. Observed NIR photoluminescence of fluorescent magnetic nanocomposites supports their use for bioimaging. The developed HINP provides dual-imaging channels for simultaneous optical and magnetic resonance imaging. PMID:21597146

Multimodal breast cancer imaging using coregistered dynamic diffuse optical tomography and digital breast tomosynthesis

NASA Astrophysics Data System (ADS)

Zimmermann, Bernhard B.; Deng, Bin; Singh, Bhawana; Martino, Mark; Selb, Juliette; Fang, Qianqian; Sajjadi, Amir Y.; Cormier, Jayne; Moore, Richard H.; Kopans, Daniel B.; Boas, David A.; Saksena, Mansi A.; Carp, Stefan A.

2017-04-01

Diffuse optical tomography (DOT) is emerging as a noninvasive functional imaging method for breast cancer diagnosis and neoadjuvant chemotherapy monitoring. In particular, the multimodal approach of combining DOT with x-ray digital breast tomosynthesis (DBT) is especially synergistic as DBT prior information can be used to enhance the DOT reconstruction. DOT, in turn, provides a functional information overlay onto the mammographic images, increasing sensitivity and specificity to cancer pathology. We describe a dynamic DOT apparatus designed for tight integration with commercial DBT scanners and providing a fast (up to 1 Hz) image acquisition rate to enable tracking hemodynamic changes induced by the mammographic breast compression. The system integrates 96 continuous-wave and 24 frequency-domain source locations as well as 32 continuous wave and 20 frequency-domain detection locations into low-profile plastic plates that can easily mate to the DBT compression paddle and x-ray detector cover, respectively. We demonstrate system performance using static and dynamic tissue-like phantoms as well as in vivo images acquired from the pool of patients recalled for breast biopsies at the Massachusetts General Hospital Breast Imaging Division.

Multifunctional PHPMA-Derived Polymer for Ratiometric pH Sensing, Fluorescence Imaging, and Magnetic Resonance Imaging.

PubMed

Su, Fengyu; Agarwal, Shubhangi; Pan, Tingting; Qiao, Yuan; Zhang, Liqiang; Shi, Zhengwei; Kong, Xiangxing; Day, Kevin; Chen, Meiwan; Meldrum, Deirdre; Kodibagkar, Vikram D; Tian, Yanqing

2018-01-17

In this paper, we report synthesis and characterization of a novel multimodality (MRI/fluorescence) probe for pH sensing and imaging. A multifunctional polymer was derived from poly(N-(2-hydroxypropyl)methacrylamide) (PHPMA) and integrated with a naphthalimide-based-ratiometric fluorescence probe and a gadolinium-1,4,7,10-tetraazacyclododecane-1,4,7,10-tetraacetic acid complex (Gd-DOTA complex). The polymer was characterized using UV-vis absorption spectrophotometry, fluorescence spectrofluorophotometry, magnetic resonance imaging (MRI), and confocal microscopy for optical and MRI-based pH sensing and cellular imaging. In vitro labeling of macrophage J774 and esophageal CP-A cell lines shows the polymer's ability to be internalized in the cells. The transverse relaxation time (T 2 ) of the polymer was observed to be pH-dependent, whereas the spin-lattice relaxation time (T 1 ) was not. The pH probe in the polymer shows a strong fluorescence-based ratiometric pH response with emission window changes, exhibiting blue emission under acidic conditions and green emission under basic conditions, respectively. This study provides new materials with multimodalities for pH sensing and imaging.

Multimodal computational microscopy based on transport of intensity equation

NASA Astrophysics Data System (ADS)

Li, Jiaji; Chen, Qian; Sun, Jiasong; Zhang, Jialin; Zuo, Chao

2016-12-01

Transport of intensity equation (TIE) is a powerful tool for phase retrieval and quantitative phase imaging, which requires intensity measurements only at axially closely spaced planes without a separate reference beam. It does not require coherent illumination and works well on conventional bright-field microscopes. The quantitative phase reconstructed by TIE gives valuable information that has been encoded in the complex wave field by passage through a sample of interest. Such information may provide tremendous flexibility to emulate various microscopy modalities computationally without requiring specialized hardware components. We develop a requisite theory to describe such a hybrid computational multimodal imaging system, which yields quantitative phase, Zernike phase contrast, differential interference contrast, and light field moment imaging, simultaneously. It makes the various observations for biomedical samples easy. Then we give the experimental demonstration of these ideas by time-lapse imaging of live HeLa cell mitosis. Experimental results verify that a tunable lens-based TIE system, combined with the appropriate postprocessing algorithm, can achieve a variety of promising imaging modalities in parallel with the quantitative phase images for the dynamic study of cellular processes.

Deformable image registration for multimodal lung-cancer staging

NASA Astrophysics Data System (ADS)

Cheirsilp, Ronnarit; Zang, Xiaonan; Bascom, Rebecca; Allen, Thomas W.; Mahraj, Rickhesvar P. M.; Higgins, William E.

2016-03-01

Positron emission tomography (PET) and X-ray computed tomography (CT) serve as major diagnostic imaging modalities in the lung-cancer staging process. Modern scanners provide co-registered whole-body PET/CT studies, collected while the patient breathes freely, and high-resolution chest CT scans, collected under a brief patient breath hold. Unfortunately, no method exists for registering a PET/CT study into the space of a high-resolution chest CT scan. If this could be done, vital diagnostic information offered by the PET/CT study could be brought seamlessly into the procedure plan used during live cancer-staging bronchoscopy. We propose a method for the deformable registration of whole-body PET/CT data into the space of a high-resolution chest CT study. We then demonstrate its potential for procedure planning and subsequent use in multimodal image-guided bronchoscopy.

Multimodality imaging of the orbit

PubMed Central

Hande, Pradipta C; Talwar, Inder

2012-01-01

The role of imaging is well established in the evaluation of orbital diseases. Ultrasonography, Computed tomography and Magnetic resonance imaging are complementary modalities, which allow direct visualization of regional anatomy, accurate localization and help to characterize lesions to make a reliable radiological diagnosis. The purpose of this pictorial essay is to highlight the imaging features of commonly encountered pathologies which involve the orbit. PMID:23599570

[Fusion of MRI, fMRI and intraoperative MRI data. Methods and clinical significance exemplified by neurosurgical interventions].

PubMed

Moche, M; Busse, H; Dannenberg, C; Schulz, T; Schmitgen, A; Trantakis, C; Winkler, D; Schmidt, F; Kahn, T

2001-11-01

The aim of this work was to realize and clinically evaluate an image fusion platform for the integration of preoperative MRI and fMRI data into the intraoperative images of an interventional MRI system with a focus on neurosurgical procedures. A vertically open 0.5 T MRI scanner was equipped with a dedicated navigation system enabling the registration of additional imaging modalities (MRI, fMRI, CT) with the intraoperatively acquired data sets. These merged image data served as the basis for interventional planning and multimodal navigation. So far, the system has been used in 70 neurosurgical interventions (13 of which involved image data fusion--requiring 15 minutes extra time). The augmented navigation system is characterized by a higher frame rate and a higher image quality as compared to the system-integrated navigation based on continuously acquired (near) real time images. Patient movement and tissue shifts can be immediately detected by monitoring the morphological differences between both navigation scenes. The multimodal image fusion allowed a refined navigation planning especially for the resection of deeply seated brain lesions or pathologies close to eloquent areas. Augmented intraoperative orientation and instrument guidance improve the safety and accuracy of neurosurgical interventions.

Research-oriented image registry for multimodal image integration.

PubMed

Tanaka, M; Sadato, N; Ishimori, Y; Yonekura, Y; Yamashita, Y; Komuro, H; Hayahsi, N; Ishii, Y

1998-01-01

To provide multimodal biomedical images automatically, we constructed the research-oriented image registry, Data Delivery System (DDS). DDS was constructed on the campus local area network. Machines which generate images (imagers: DSA, ultrasound, PET, MRI, SPECT and CT) were connected to the campus LAN. Once a patient is registered, all his images are automatically picked up by DDS as they are generated, transferred through the gateway server to the intermediate server, and copied into the directory of the user who registered the patient. DDS informs the user through e-mail that new data have been generated and transferred. Data format is automatically converted into one which is chosen by the user. Data inactive for a certain period in the intermediate server are automatically achieved into the final and permanent data server based on compact disk. As a soft link is automatically generated through this step, a user has access to all (old or new) image data of the patient of his interest. As DDS runs with minimal maintenance, cost and time for data transfer are significantly saved. By making the complex process of data transfer and conversion invisible, DDS has made it easy for naive-to-computer researchers to concentrate on their biomedical interest.

An on-chip silicon compact triplexer based on cascaded tilted multimode interference couplers

NASA Astrophysics Data System (ADS)

Chen, Jingye; Liu, Penghao; Shi, Yaocheng

2018-03-01

An on-chip triplexer based on cascaded tilted multimode interference (MMI) couplers has been demonstrated to separate the 1310 nm wavelength band into one port and 1490 nm and 1550 nm wavelength bands into the other two ports respectively. By utilizing the dispersive self-imaging and pseudo self-imaging, the device length is not critically determined by the common multiple of beat lengths for different wavelengths. The total device size can be reduced to ∼450 μm, which is half of the butterfly structure reported. The whole device, fabricated with only one fully-etching step, is characterized with <-15 dB low crosstalk (CT) and ∼1 dB insertion loss (IL).

Multimodal imaging of spike propagation: a technical case report.

PubMed

Tanaka, N; Grant, P E; Suzuki, N; Madsen, J R; Bergin, A M; Hämäläinen, M S; Stufflebeam, S M

2012-06-01

We report an 11-year-old boy with intractable epilepsy, who had cortical dysplasia in the right superior frontal gyrus. Spatiotemporal source analysis of MEG and EEG spikes demonstrated a similar time course of spike propagation from the superior to inferior frontal gyri, as observed on intracranial EEG. The tractography reconstructed from DTI showed a fiber connection between these areas. Our multimodal approach demonstrates spike propagation and a white matter tract guiding the propagation.

Integrative, multimodal analysis of glioblastoma using TCGA molecular data, pathology images, and clinical outcomes.

PubMed

Kong, Jun; Cooper, Lee A D; Wang, Fusheng; Gutman, David A; Gao, Jingjing; Chisolm, Candace; Sharma, Ashish; Pan, Tony; Van Meir, Erwin G; Kurc, Tahsin M; Moreno, Carlos S; Saltz, Joel H; Brat, Daniel J

2011-12-01

Multimodal, multiscale data synthesis is becoming increasingly critical for successful translational biomedical research. In this letter, we present a large-scale investigative initiative on glioblastoma, a high-grade brain tumor, with complementary data types using in silico approaches. We integrate and analyze data from The Cancer Genome Atlas Project on glioblastoma that includes novel nuclear phenotypic data derived from microscopic slides, genotypic signatures described by transcriptional class and genetic alterations, and clinical outcomes defined by response to therapy and patient survival. Our preliminary results demonstrate numerous clinically and biologically significant correlations across multiple data types, revealing the power of in silico multimodal data integration for cancer research.

Multimodal fiber source for nonlinear microscopy based on a dissipative soliton laser

PubMed Central

Lamb, Erin S.; Wise, Frank W.

2015-01-01

Recent developments in high energy femtosecond fiber lasers have enabled robust and lower-cost sources for multiphoton-fluorescence and harmonic-generation imaging. However, picosecond pulses are better suited for Raman scattering microscopy, so the ideal multimodal source for nonlinear microcopy needs to provide both durations. Here we present spectral compression of a high-power femtosecond fiber laser as a route to producing transform-limited picosecond pulses. These pulses pump a fiber optical parametric oscillator to yield a robust fiber source capable of providing the synchronized picosecond pulse trains needed for Raman scattering microscopy. Thus, this system can be used as a multimodal platform for nonlinear microscopy techniques. PMID:26417497

Hierarchical storage of large volume of multidector CT data using distributed servers

NASA Astrophysics Data System (ADS)

Ratib, Osman; Rosset, Antoine; Heuberger, Joris; Bandon, David

2006-03-01

Multidector scanners and hybrid multimodality scanners have the ability to generate large number of high-resolution images resulting in very large data sets. In most cases, these datasets are generated for the sole purpose of generating secondary processed images and 3D rendered images as well as oblique and curved multiplanar reformatted images. It is therefore not essential to archive the original images after they have been processed. We have developed an architecture of distributed archive servers for temporary storage of large image datasets for 3D rendering and image processing without the need for long term storage in PACS archive. With the relatively low cost of storage devices it is possible to configure these servers to hold several months or even years of data, long enough for allowing subsequent re-processing if required by specific clinical situations. We tested the latest generation of RAID servers provided by Apple computers with a capacity of 5 TBytes. We implemented a peer-to-peer data access software based on our Open-Source image management software called OsiriX, allowing remote workstations to directly access DICOM image files located on the server through a new technology called "bonjour". This architecture offers a seamless integration of multiple servers and workstations without the need for central database or complex workflow management tools. It allows efficient access to image data from multiple workstation for image analysis and visualization without the need for image data transfer. It provides a convenient alternative to centralized PACS architecture while avoiding complex and time-consuming data transfer and storage.

Multimodal image registration based on binary gradient angle descriptor.

PubMed

Jiang, Dongsheng; Shi, Yonghong; Yao, Demin; Fan, Yifeng; Wang, Manning; Song, Zhijian

2017-12-01

Multimodal image registration plays an important role in image-guided interventions/therapy and atlas building, and it is still a challenging task due to the complex intensity variations in different modalities. The paper addresses the problem and proposes a simple, compact, fast and generally applicable modality-independent binary gradient angle descriptor (BGA) based on the rationale of gradient orientation alignment. The BGA can be easily calculated at each voxel by coding the quadrant in which a local gradient vector falls, and it has an extremely low computational complexity, requiring only three convolutions, two multiplication operations and two comparison operations. Meanwhile, the binarized encoding of the gradient orientation makes the BGA more resistant to image degradations compared with conventional gradient orientation methods. The BGA can extract similar feature descriptors for different modalities and enable the use of simple similarity measures, which makes it applicable within a wide range of optimization frameworks. The results for pairwise multimodal and monomodal registrations between various images (T1, T2, PD, T1c, Flair) consistently show that the BGA significantly outperforms localized mutual information. The experimental results also confirm that the BGA can be a reliable alternative to the sum of absolute difference in monomodal image registration. The BGA can also achieve an accuracy of [Formula: see text], similar to that of the SSC, for the deformable registration of inhale and exhale CT scans. Specifically, for the highly challenging deformable registration of preoperative MRI and 3D intraoperative ultrasound images, the BGA achieves a similar registration accuracy of [Formula: see text] compared with state-of-the-art approaches, with a computation time of 18.3 s per case. The BGA improves the registration performance in terms of both accuracy and time efficiency. With further acceleration, the framework has the potential for application in time-sensitive clinical environments, such as for preoperative MRI and intraoperative US image registration for image-guided intervention.

Computer-assisted surgical planning and automation of laser delivery systems

NASA Astrophysics Data System (ADS)

Zamorano, Lucia J.; Dujovny, Manuel; Dong, Ada; Kadi, A. Majeed

1991-05-01

This paper describes a 'real time' surgical treatment planning interactive workstation, utilizing multimodality imaging (computer tomography, magnetic resonance imaging, digital angiography) that has been developed to provide the neurosurgeon with two-dimensional multiplanar and three-dimensional 'display' of a patient's lesion.

Novel DOTA-based prochelator for divalent peptide vectorization: synthesis of dimeric bombesin analogues for multimodality tumor imaging and therapy.

PubMed

Abiraj, Keelara; Jaccard, Hugues; Kretzschmar, Martin; Helm, Lothar; Maecke, Helmut R

2008-07-28

Dimeric peptidic vectors, obtained by the divalent grafting of bombesin analogues on a newly synthesized DOTA-based prochelator, showed improved qualities as tumor targeted imaging probes in comparison to their monomeric analogues.

In vivo evaluation of adipose- and muscle-derived stem cells as a treatment for nonhealing diabetic wounds using multimodal microscopy

NASA Astrophysics Data System (ADS)

Li, Joanne; Pincu, Yair; Marjanovic, Marina; Bower, Andrew J.; Chaney, Eric J.; Jensen, Tor; Boppart, Marni D.; Boppart, Stephen A.

2016-08-01

Impaired skin wound healing is a significant comorbid condition of diabetes, which often results in nonhealing diabetic ulcers due to poor peripheral microcirculation, among other factors. The effectiveness of the regeneration of adipose-derived stem cells (ADSCs) and muscle-derived stem cells (MDSCs) was assessed using an integrated multimodal microscopy system equipped with two-photon fluorescence and second-harmonic generation imaging. These imaging modalities, integrated in a single platform for spatial and temporal coregistration, allowed us to monitor in vivo changes in the collagen network and cell dynamics in a skin wound. Fluorescently labeled ADSCs and MDSCs were applied topically to the wound bed of wild-type and diabetic (db/db) mice following punch biopsy. Longitudinal imaging demonstrated that ADSCs and MDSCs provided remarkable capacity for improved diabetic wound healing, and integrated microscopy revealed a more organized collagen remodeling in the wound bed of treated mice. The results from this study verify the regenerative capacity of stem cells toward healing and, with multimodal microscopy, provide insight regarding their impact on the skin microenvironment. The optical method outlined in this study, which has the potential for in vivo human use, may optimize the care and treatment of diabetic nonhealing wounds.

Mixture-Tuned, Clutter Matched Filter for Remote Detection of Subpixel Spectral Signals

NASA Technical Reports Server (NTRS)

Thompson, David R.; Mandrake, Lukas; Green, Robert O.

2013-01-01

Mapping localized spectral features in large images demands sensitive and robust detection algorithms. Two aspects of large images that can harm matched-filter detection performance are addressed simultaneously. First, multimodal backgrounds may thwart the typical Gaussian model. Second, outlier features can trigger false detections from large projections onto the target vector. Two state-of-the-art approaches are combined that independently address outlier false positives and multimodal backgrounds. The background clustering models multimodal backgrounds, and the mixture tuned matched filter (MT-MF) addresses outliers. Combining the two methods captures significant additional performance benefits. The resulting mixture tuned clutter matched filter (MT-CMF) shows effective performance on simulated and airborne datasets. The classical MNF transform was applied, followed by k-means clustering. Then, each cluster s mean, covariance, and the corresponding eigenvalues were estimated. This yields a cluster-specific matched filter estimate as well as a cluster- specific feasibility score to flag outlier false positives. The technology described is a proof of concept that may be employed in future target detection and mapping applications for remote imaging spectrometers. It is of most direct relevance to JPL proposals for airborne and orbital hyperspectral instruments. Applications include subpixel target detection in hyperspectral scenes for military surveillance. Earth science applications include mineralogical mapping, species discrimination for ecosystem health monitoring, and land use classification.

Rotational electrical impedance tomography using electrodes with limited surface coverage provides window for multimodal sensing

NASA Astrophysics Data System (ADS)

Lehti-Polojärvi, Mari; Koskela, Olli; Seppänen, Aku; Figueiras, Edite; Hyttinen, Jari

2018-02-01

Electrical impedance tomography (EIT) is an imaging method that could become a valuable tool in multimodal applications. One challenge in simultaneous multimodal imaging is that typically the EIT electrodes cover a large portion of the object surface. This paper investigates the feasibility of rotational EIT (rEIT) in applications where electrodes cover only a limited angle of the surface of the object. In the studied rEIT, the object is rotated a full 360° during a set of measurements to increase the information content of the data. We call this approach limited angle full revolution rEIT (LAFR-rEIT). We test LAFR-rEIT setups in two-dimensional geometries with computational and experimental data. We use up to 256 rotational measurement positions, which requires a new way to solve the forward and inverse problem of rEIT. For this, we provide a modification, available for EIDORS, in the supplementary material. The computational results demonstrate that LAFR-rEIT with eight electrodes produce the same image quality as conventional 16-electrode rEIT, when data from an adequate number of rotational measurement positions are used. Both computational and experimental results indicate that the novel LAFR-rEIT provides good EIT with setups with limited surface coverage and a small number of electrodes.

Multimodal MRI in cerebral small vessel disease: its relationship with cognition and sensitivity to change over time.

PubMed

Nitkunan, Arani; Barrick, Tom R; Charlton, Rebecca A; Clark, Chris A; Markus, Hugh S

2008-07-01

Cerebral small vessel disease is the most common cause of vascular dementia. Interest in using MRI parameters as surrogate markers of disease to assess therapies is increasing. In patients with symptomatic sporadic small vessel disease, we determined which MRI parameters best correlated with cognitive function on cross-sectional analysis and which changed over a period of 1 year. Thirty-five patients with lacunar stroke and leukoaraiosis were recruited. They underwent multimodal MRI (brain volume, fluid-attenuated inversion recovery lesion load, lacunar infarct number, fractional anisotropy, and mean diffusivity from diffusion tensor imaging) and neuropsychological testing. Twenty-seven agreed to reattend for repeat MRI and neuropsychology at 1 year. An executive function score correlated most strongly with diffusion tensor imaging (fractional anisotropy histogram, r=-0.640, P=0.004) and brain volume (r=0.501, P=0.034). Associations with diffusion tensor imaging were stronger than with all other MRI parameters. On multiple regression of all imaging parameters, a model that contained brain volume and fractional anisotropy, together with age, gender, and premorbid IQ, explained 74% of the variance of the executive function score (P=0.0001). Changes in mean diffusivity and fractional anisotropy were detectable over the 1-year follow-up; in contrast, no change in other MRI parameters was detectable over this time period. A multimodal MRI model explains a large proportion of the variation in executive function in cerebral small vessel disease. In particular, diffusion tensor imaging correlates best with executive function and is the most sensitive to change. This supports the use of MRI, in particular diffusion tensor imaging, as a surrogate marker in treatment trials.

Analyzing multimodality tomographic images and associated regions of interest with MIDAS

NASA Astrophysics Data System (ADS)

Tsui, Wai-Hon; Rusinek, Henry; Van Gelder, Peter; Lebedev, Sergey

2001-07-01

This paper outlines the design and features incorporated in a software package for analyzing multi-modality tomographic images. The package MIDAS has been evolving for the past 15 years and is in wide use by researchers at New York University School of Medicine and a number of collaborating research sites. It was written in the C language and runs on Sun workstations and Intel PCs under the Solaris operating system. A unique strength of the MIDAS package lies in its ability to generate, manipulate and analyze a practically unlimited number of regions of interest (ROIs). These regions are automatically saved in an efficient data structure and linked to associated images. A wide selection of set theoretical (e.g. union, xor, difference), geometrical (e.g. move, rotate) and morphological (grow, peel) operators can be applied to an arbitrary selection of ROIs. ROIs are constructed as a result of image segmentation algorithms incorporated in MIDAS; they also can be drawn interactively. These ROI editing operations can be applied in either 2D or 3D mode. ROI statistics generated by MIDAS include means, standard deviations, centroids and histograms. Other image manipulation tools incorporated in MIDAS are multimodality and within modality coregistration methods (including landmark matching, surface fitting and Woods' correlation methods) and image reformatting methods (using nearest-neighbor, tri-linear or sinc interpolation). Applications of MIDAS include: (1) neuroanatomy research: marking anatomical structures in one orientation, reformatting marks to another orientation; (2) tissue volume measurements: brain structures (PET, MRI, CT), lung nodules (low dose CT), breast density (MRI); (3) analysis of functional (SPECT, PET) experiments by overlaying corresponding structural scans; (4) longitudinal studies: regional measurement of atrophy.

Clinical application of a multi-modal endoscope with fluorescent peptide in detection of colorectal neoplasia (Conference Presentation)

NASA Astrophysics Data System (ADS)

Dai, Zhenzhen; Joshi, Bishnu P.; Gao, Zhenghong; Lee, Jeonghoon; Ghimire, Navin; Prabhu, Anoop; Wamsteker, Erik J.; Kwon, Richard S.; Elta, Grace H.; Appelman, Henry D.; Owens, Scott R.; Kuick, Rork; Turgeon, Kim K.; Wang, Thomas D.

2017-02-01

Early detection of precursor lesions for colorectal cancer can greatly improve survival. Pre-neoplasia can appear flat with conventional white light endoscopy. Sessile serrated adenomas (SSA) are precursor lesions found primarily in the proximal colon and frequently appear flat and indistinct. We performed a clinical study of n=37 patients using a multimodal endoscopy with a FITC-labeled peptide specific for SSA. Lesions were imaged with white light, reflectance and fluorescence. White light images were acquired before the peptide was applied and were used to help localize regions of abnormal tissues rightly. Co-registered fluorescence and reflectance images were combined to get ratio images thus the distance was corrected. We calculated the target/background ratio (T/B ratio) to quantify the images and found 2.3-fold greater fluorescence intensity for SSA compared with normal tissues. We found the T/B ratio for SSA to be significantly greater than that for normal colonic mucosa with 89.47% sensitivity and 91.67% specificity at the threshold of 1.22. An ROC curve for SSA and normal mucosa was also plotted with area under curve (AUC) of 0.93. The result also shows that SSA and adenoma are statistically significant and can be identified with 78.95% sensitivity and 90.48% specificity at the threshold of 1.66. An ROC curve was plotted with AUC of 0.88. Therefore, our result shows that the application of a multimodal endoscope with fluorescently labeled peptide can quantify images and works especially good for the detection of SSA which is a premalignant flat lesion conferring a high risk of subsequently leading to a colon cancer.

In vivo features of melanocytic lesions: multimode hyperspectral dermoscopy, reflectance confocal microscopy, and histopathologic correlates (Conference Presentation)

NASA Astrophysics Data System (ADS)

Vasefi, Fartash; MacKinnon, Nicholas B.; Jain, Manu; Cordova, Miguel A.; Kose, Kivanc; Rajadhyaksha, Milind; Halpern, Allan C.; Farkas, Daniel L.

2017-02-01

Motivation and background: Melanoma, the fastest growing cancer worldwide, kills more than one person every hour in the United States. Determining the depth and distribution of dermal melanin and hemoglobin adds physio-morphologic information to the current diagnostic standard, cellular morphology, to further develop noninvasive methods to discriminate between melanoma and benign skin conditions. Purpose: To compare the performance of a multimode dermoscopy system (SkinSpect), which is designed to quantify and map in three dimensions, in vivo melanin and hemoglobin in skin, and to validate this with histopathology and three dimensional reflectance confocal microscopy (RCM) imaging. Methods: Sequentially capture SkinSpect and RCM images of suspect lesions and nearby normal skin and compare this with histopathology reports, RCM imaging allows noninvasive observation of nuclear, cellular and structural detail in 1-5 μm-thin optical sections in skin, and detection of pigmented skin lesions with sensitivity of 90-95% and specificity of 70-80%. The multimode imaging dermoscope combines polarization (cross and parallel), autofluorescence and hyperspectral imaging to noninvasively map the distribution of melanin, collagen and hemoglobin oxygenation in pigmented skin lesions. Results: We compared in vivo features of ten melanocytic lesions extracted by SkinSpect and RCM imaging, and correlated them to histopathologic results. We present results of two melanoma cases (in situ and invasive), and compare with in vivo features from eight benign lesions. Melanin distribution at different depths and hemodynamics, including abnormal vascularity, detected by both SkinSpect and RCM will be discussed. Conclusion: Diagnostic features such as dermal melanin and hemoglobin concentration provided in SkinSpect skin analysis for melanoma and normal pigmented lesions can be compared and validated using results from RCM and histopathology.

DETECTION OF TYPE 1 CHOROIDAL NEOVASCULAR MEMBRANES USING OPTICAL COHERENCE TOMOGRAPHY ANGIOGRAPHY IN TUBERCULAR POSTERIOR UVEITIS.

PubMed

Aggarwal, Kanika; Agarwal, Aniruddha; Sharma, Aman; Sharma, Kusum; Gupta, Vishali

2018-04-23

To study optical coherence tomography angiography (OCTA) and multimodal imaging features of Type 1 inflammatory choroidal neovascularization (CNV) in tubercular serpiginous-like choroiditis and response to anti-vascular endothelial growth factor therapy. In this study, multimodal imaging was performed using OCTA, enhanced-depth imaging optical coherence tomography, fluorescein angiography, and indocyanine green angiography. Correlation of OCTA with other imaging modalities in the detection of CNV was performed. The changes in CNV configuration after anti-vascular endothelial growth factor therapy were assessed. In this study, nine eyes (8 patients; 5 females; mean age: 32.5 ± 11.57 years) with diagnosis of tubercular serpiginous-like choroiditis were included. All the eyes had presence of low-lying pigment epithelial detachments on enhanced-depth imaging optical coherence tomography. Using OCTA, it was possible to detect Type 1 CNV in all eyes. Type 1 CNV networks comprised fine anastomotic network of vessels, some of which had a hairpin loop configuration. After anti-vascular endothelial growth factor therapy, there was a decrease in branching and anastomosis. The visual acuity significantly improved from 0.49 ± 0.26 (20/60 Snellen equivalent) at baseline to 0.26 ± 0.17 (20/36 Snellen equivalent) (P = 0.03) in all eyes. Type 1 CNV can occur among patients with tubercular serpiginous-like choroiditis, leading to significant visual loss even in the healed stages of the disease. Optical coherence tomography angiography can help in the detection of Type 1 CNV where conventional multimodal imaging, including fluorescein angiography and OCT, fails to make a definitive diagnosis and thereby guide the initiation of anti-vascular endothelial growth factor therapy.

A multimodal image guiding system for Navigated Ultrasound Bronchoscopy (EBUS): A human feasibility study

PubMed Central

Hofstad, Erlend Fagertun; Amundsen, Tore; Langø, Thomas; Bakeng, Janne Beate Lervik; Leira, Håkon Olav

2017-01-01

Background Endobronchial ultrasound transbronchial needle aspiration (EBUS-TBNA) is the endoscopic method of choice for confirming lung cancer metastasis to mediastinal lymph nodes. Precision is crucial for correct staging and clinical decision-making. Navigation and multimodal imaging can potentially improve EBUS-TBNA efficiency. Aims To demonstrate the feasibility of a multimodal image guiding system using electromagnetic navigation for ultrasound bronchoschopy in humans. Methods Four patients referred for lung cancer diagnosis and staging with EBUS-TBNA were enrolled in the study. Target lymph nodes were predefined from the preoperative computed tomography (CT) images. A prototype convex probe ultrasound bronchoscope with an attached sensor for position tracking was used for EBUS-TBNA. Electromagnetic tracking of the ultrasound bronchoscope and ultrasound images allowed fusion of preoperative CT and intraoperative ultrasound in the navigation software. Navigated EBUS-TBNA was used to guide target lymph node localization and sampling. Navigation system accuracy was calculated, measured by the deviation between lymph node position in ultrasound and CT in three planes. Procedure time, diagnostic yield and adverse events were recorded. Results Preoperative CT and real-time ultrasound images were successfully fused and displayed in the navigation software during the procedures. Overall navigation accuracy (11 measurements) was 10.0 ± 3.8 mm, maximum 17.6 mm, minimum 4.5 mm. An adequate sample was obtained in 6/6 (100%) of targeted lymph nodes. No adverse events were registered. Conclusions Electromagnetic navigated EBUS-TBNA was feasible, safe and easy in this human pilot study. The clinical usefulness was clearly demonstrated. Fusion of real-time ultrasound, preoperative CT and electromagnetic navigational bronchoscopy provided a controlled guiding to level of target, intraoperative overview and procedure documentation. PMID:28182758

Nanoparticles in magnetic resonance imaging: from simple to dual contrast agents

PubMed Central

Estelrich, Joan; Sánchez-Martín, María Jesús; Busquets, Maria Antònia

2015-01-01

Magnetic resonance imaging (MRI) has become one of the most widely used and powerful tools for noninvasive clinical diagnosis owing to its high degree of soft tissue contrast, spatial resolution, and depth of penetration. MRI signal intensity is related to the relaxation times (T1, spin–lattice relaxation and T2, spin–spin relaxation) of in vivo water protons. To increase contrast, various inorganic nanoparticles and complexes (the so-called contrast agents) are administered prior to the scanning. Shortening T1 and T2 increases the corresponding relaxation rates, 1/T1 and 1/T2, producing hyperintense and hypointense signals respectively in shorter times. Moreover, the signal-to-noise ratio can be improved with the acquisition of a large number of measurements. The contrast agents used are generally based on either iron oxide nanoparticles or ferrites, providing negative contrast in T2-weighted images; or complexes of lanthanide metals (mostly containing gadolinium ions), providing positive contrast in T1-weighted images. Recently, lanthanide complexes have been immobilized in nanostructured materials in order to develop a new class of contrast agents with functions including blood-pool and organ (or tumor) targeting. Meanwhile, to overcome the limitations of individual imaging modalities, multimodal imaging techniques have been developed. An important challenge is to design all-in-one contrast agents that can be detected by multimodal techniques. Magnetoliposomes are efficient multimodal contrast agents. They can simultaneously bear both kinds of contrast and can, furthermore, incorporate targeting ligands and chains of polyethylene glycol to enhance the accumulation of nanoparticles at the site of interest and the bioavailability, respectively. Here, we review the most important characteristics of the nanoparticles or complexes used as MRI contrast agents. PMID:25834422

Comparison of the application of B-mode and strain elastography ultrasound in the estimation of lymph node metastasis of papillary thyroid carcinoma based on a radiomics approach.

PubMed

Liu, Tongtong; Ge, Xifeng; Yu, Jinhua; Guo, Yi; Wang, Yuanyuan; Wang, Wenping; Cui, Ligang

2018-06-21

B-mode ultrasound (B-US) and strain elastography ultrasound (SE-US) images have a potential to distinguish thyroid tumor with different lymph node (LN) status. The purpose of our study is to investigate whether the application of multi-modality images including B-US and SE-US can improve the discriminability of thyroid tumor with LN metastasis based on a radiomics approach. Ultrasound (US) images including B-US and SE-US images of 75 papillary thyroid carcinoma (PTC) cases were retrospectively collected. A radiomics approach was developed in this study to estimate LNs status of PTC patients. The approach included image segmentation, quantitative feature extraction, feature selection and classification. Three feature sets were extracted from B-US, SE-US, and multi-modality containing B-US and SE-US. They were used to evaluate the contribution of different modalities. A total of 684 radiomics features have been extracted in our study. We used sparse representation coefficient-based feature selection method with 10-bootstrap to reduce the dimension of feature sets. Support vector machine with leave-one-out cross-validation was used to build the model for estimating LN status. Using features extracted from both B-US and SE-US, the radiomics-based model produced an area under the receiver operating characteristic curve (AUC) [Formula: see text] 0.90, accuracy (ACC) [Formula: see text] 0.85, sensitivity (SENS) [Formula: see text] 0.77 and specificity (SPEC) [Formula: see text] 0.88, which was better than using features extracted from B-US or SE-US separately. Multi-modality images provided more information in radiomics study. Combining use of B-US and SE-US could improve the LN metastasis estimation accuracy for PTC patients.

Multimodality Imaging of RNA Interference

PubMed Central

Nayak, Tapas R.; Krasteva, Lazura K.; Cai, Weibo

2013-01-01

The discovery of small interfering RNAs (siRNAs) and their potential to knock down virtually any gene of interest has ushered in a new era of RNA interference (RNAi). Clinical use of RNAi faces severe limitations due to inefficiency delivery of siRNA or short hairpin RNA (shRNA). Many molecular imaging techniques have been adopted in RNAi-related research for evaluation of siRNA/shRNA delivery, biodistribution, pharmacokinetics, and the therapeutic effect. In this review article, we summarize the current status of in vivo imaging of RNAi. The molecular imaging techniques that have been employed include bioluminescence/fluorescence imaging, magnetic resonance imaging/spectroscopy, positron emission tomography, single-photon emission computed tomography, and various combinations of these techniques. Further development of non-invasive imaging strategies for RNAi, not only focusing on the delivery of siRNA/shRNA but also the therapeutic efficacy, is critical for future clinical translation. Rigorous validation will be needed to confirm that biodistribution of the carrier is correlated with that of siRNA/shRNA, since imaging only detects the label (e.g. radioisotopes) but not the gene or carrier themselves. It is also essential to develop multimodality imaging approaches for realizing the full potential of therapeutic RNAi, as no single imaging modality may be sufficient to simultaneously monitor both the gene delivery and silencing effect of RNAi. PMID:23745567

Design of high energy laser pulse delivery in a multimode fiber for photoacoustic tomography.

PubMed

Ai, Min; Shu, Weihang; Salcudean, Tim; Rohling, Robert; Abolmaesumi, Purang; Tang, Shuo

2017-07-24

In photoacoustic tomography (PAT), delivering high energy pulses through optical fiber is critical for achieving high quality imaging. A fiber coupling scheme with a beam homogenizer is demonstrated for coupling high energy pulses in a single multimode fiber. This scheme can benefit PAT applications that require miniaturized illumination or internal illumination with a small fiber. The beam homogenizer is achieved by using a cross cylindrical lens array, which provides a periodic spatial modulation on the phase of the input light. Thus the lens array acts as a phase grating which diffracts the beam into a 2D diffraction pattern. Both theoretical analysis and experiments demonstrate that the focused beam can be split into a 2D spot array that can reduce the peak power on the fiber tip surface and thus enhance the coupling performance. The theoretical analysis of the intensity distribution of the focused beam is carried out by Fourier optics. In experiments, coupled energy at 48 mJ/pulse and 60 mJ/pulse have been achieved and the corresponding coupling efficiency is 70% and 90% in a 1000-μm and a 1500-μm-core-diameter fiber, respectively. The high energy pulses delivered by the multimode fiber are further tested for PAT imaging in phantoms. PAT imaging of a printed dot array shows a large illumination area of 7 cm 2 under 5 mm thick chicken breast tissue. In vivo imaging is also demonstrated on the human forearm. The large improvement in coupling energy can potentially benefit PAT with single fiber delivery to achieve large area imaging and deep penetration detection.

Cryo-Imaging and Software Platform for Analysis of Molecular MR Imaging of Micrometastases

PubMed Central

Qutaish, Mohammed Q.; Zhou, Zhuxian; Prabhu, David; Liu, Yiqiao; Busso, Mallory R.; Izadnegahdar, Donna; Gargesha, Madhusudhana; Lu, Hong; Lu, Zheng-Rong

2018-01-01

We created and evaluated a preclinical, multimodality imaging, and software platform to assess molecular imaging of small metastases. This included experimental methods (e.g., GFP-labeled tumor and high resolution multispectral cryo-imaging), nonrigid image registration, and interactive visualization of imaging agent targeting. We describe technological details earlier applied to GFP-labeled metastatic tumor targeting by molecular MR (CREKA-Gd) and red fluorescent (CREKA-Cy5) imaging agents. Optimized nonrigid cryo-MRI registration enabled nonambiguous association of MR signals to GFP tumors. Interactive visualization of out-of-RAM volumetric image data allowed one to zoom to a GFP-labeled micrometastasis, determine its anatomical location from color cryo-images, and establish the presence/absence of targeted CREKA-Gd and CREKA-Cy5. In a mouse with >160 GFP-labeled tumors, we determined that in the MR images every tumor in the lung >0.3 mm2 had visible signal and that some metastases as small as 0.1 mm2 were also visible. More tumors were visible in CREKA-Cy5 than in CREKA-Gd MRI. Tape transfer method and nonrigid registration allowed accurate (<11 μm error) registration of whole mouse histology to corresponding cryo-images. Histology showed inflammation and necrotic regions not labeled by imaging agents. This mouse-to-cells multiscale and multimodality platform should uniquely enable more informative and accurate studies of metastatic cancer imaging and therapy. PMID:29805438

A Tactile Carina Nebula

NASA Astrophysics Data System (ADS)

Grice, Noreen A.; Mutchler, M.

2010-01-01

Astronomy was once considered a science restricted to fully sighted participants. But in the past two decades, accessible books with large print/Braille and touchable pictures have brought astronomy and space science to the hands and mind's eye of students, regardless of their visual ability. A new universally-designed tactile image featuring the Hubble mosaic of the Carina Nebula is being presented at this conference. The original dataset was obtained with Hubble's Advanced Camera for Surveys (ACS) hydrogen-alpha filter in 2005. It became an instant icon after being infused with additional color information from ground-based CTIO data, and released as Hubble's 17th anniversary image. Our tactile Carina Nebula promotes multi-mode learning about the entire life-cycle of stars, which is dramatically illustrated in this Hubble mosaic. When combined with descriptive text in print and Braille, the visual and tactile components seamlessly reach both sighted and blind populations. Specific touchable features of the tactile image identify the shapes and orientations of objects in the Carina Nebula that include star-forming regions, jets, pillars, dark and light globules, star clusters, shocks/bubbles, the Keyhole Nebula, and stellar death (Eta Carinae). Visit our poster paper to touch the Carina Nebula!

Radiation Safety in Children With Congenital and Acquired Heart Disease: A Scientific Position Statement on Multimodality Dose Optimization From the Image Gently Alliance.

PubMed

Hill, Kevin D; Frush, Donald P; Han, B Kelly; Abbott, Brian G; Armstrong, Aimee K; DeKemp, Robert A; Glatz, Andrew C; Greenberg, S Bruce; Herbert, Alexander Sheldon; Justino, Henri; Mah, Douglas; Mahesh, Mahadevappa; Rigsby, Cynthia K; Slesnick, Timothy C; Strauss, Keith J; Trattner, Sigal; Viswanathan, Mohan N; Einstein, Andrew J

2017-07-01

There is a need for consensus recommendations for ionizing radiation dose optimization during multimodality medical imaging in children with congenital and acquired heart disease (CAHD). These children often have complex diseases and may be exposed to a relatively high cumulative burden of ionizing radiation from medical imaging procedures, including cardiac computed tomography, nuclear cardiology studies, and fluoroscopically guided diagnostic and interventional catheterization and electrophysiology procedures. Although these imaging procedures are all essential to the care of children with CAHD and have contributed to meaningfully improved outcomes in these patients, exposure to ionizing radiation is associated with potential risks, including an increased lifetime attributable risk of cancer. The goal of these recommendations is to encourage informed imaging to achieve appropriate study quality at the lowest achievable dose. Other strategies to improve care include a patient-centered approach to imaging, emphasizing education and informed decision making and programmatic approaches to ensure appropriate dose monitoring. Looking ahead, there is a need for standardization of dose metrics across imaging modalities, so as to encourage comparative effectiveness studies across the spectrum of CAHD in children. Copyright © 2017 The Authors. Published by Elsevier Inc. All rights reserved.

Multimodal MSI in Conjunction with Broad Coverage Spatially Resolved MS2 Increases Confidence in Both Molecular Identification and Localization.

PubMed

Veličković, Dušan; Chu, Rosalie K; Carrell, Alyssa A; Thomas, Mathew; Paša-Tolić, Ljiljana; Weston, David J; Anderton, Christopher R

2018-01-02

One critical aspect of mass spectrometry imaging (MSI) is the need to confidently identify detected analytes. While orthogonal tandem MS (e.g., LC-MS 2 ) experiments from sample extracts can assist in annotating ions, the spatial information about these molecules is lost. Accordingly, this could cause mislead conclusions, especially in cases where isobaric species exhibit different distributions within a sample. In this Technical Note, we employed a multimodal imaging approach, using matrix assisted laser desorption/ionization (MALDI)-MSI and liquid extraction surface analysis (LESA)-MS 2 I, to confidently annotate and localize a broad range of metabolites involved in a tripartite symbiosis system of moss, cyanobacteria, and fungus. We found that the combination of these two imaging modalities generated very congruent ion images, providing the link between highly accurate structural information onfered by LESA and high spatial resolution attainable by MALDI. These results demonstrate how this combined methodology could be very useful in differentiating metabolite routes in complex systems.

MINC 2.0: A Flexible Format for Multi-Modal Images.

PubMed

Vincent, Robert D; Neelin, Peter; Khalili-Mahani, Najmeh; Janke, Andrew L; Fonov, Vladimir S; Robbins, Steven M; Baghdadi, Leila; Lerch, Jason; Sled, John G; Adalat, Reza; MacDonald, David; Zijdenbos, Alex P; Collins, D Louis; Evans, Alan C

2016-01-01

It is often useful that an imaging data format can afford rich metadata, be flexible, scale to very large file sizes, support multi-modal data, and have strong inbuilt mechanisms for data provenance. Beginning in 1992, MINC was developed as a system for flexible, self-documenting representation of neuroscientific imaging data with arbitrary orientation and dimensionality. The MINC system incorporates three broad components: a file format specification, a programming library, and a growing set of tools. In the early 2000's the MINC developers created MINC 2.0, which added support for 64-bit file sizes, internal compression, and a number of other modern features. Because of its extensible design, it has been easy to incorporate details of provenance in the header metadata, including an explicit processing history, unique identifiers, and vendor-specific scanner settings. This makes MINC ideal for use in large scale imaging studies and databases. It also makes it easy to adapt to new scanning sequences and modalities.

Computer-aided, multi-modal, and compression diffuse optical studies of breast tissue

NASA Astrophysics Data System (ADS)

Busch, David Richard, Jr.

Diffuse Optical Tomography and Spectroscopy permit measurement of important physiological parameters non-invasively through ˜10 cm of tissue. I have applied these techniques in measurements of human breast and breast cancer. My thesis integrates three loosely connected themes in this context: multi-modal breast cancer imaging, automated data analysis of breast cancer images, and microvascular hemodynamics of breast under compression. As per the first theme, I describe construction, testing, and the initial clinical usage of two generations of imaging systems for simultaneous diffuse optical and magnetic resonance imaging. The second project develops a statistical analysis of optical breast data from many spatial locations in a population of cancers to derive a novel optical signature of malignancy; I then apply this data-derived signature for localization of cancer in additional subjects. Finally, I construct and deploy diffuse optical instrumentation to measure blood content and blood flow during breast compression; besides optics, this research has implications for any method employing breast compression, e.g., mammography.

Distance-Dependent Multimodal Image Registration for Agriculture Tasks

PubMed Central

Berenstein, Ron; Hočevar, Marko; Godeša, Tone; Edan, Yael; Ben-Shahar, Ohad

2015-01-01

Image registration is the process of aligning two or more images of the same scene taken at different times; from different viewpoints; and/or by different sensors. This research focuses on developing a practical method for automatic image registration for agricultural systems that use multimodal sensory systems and operate in natural environments. While not limited to any particular modalities; here we focus on systems with visual and thermal sensory inputs. Our approach is based on pre-calibrating a distance-dependent transformation matrix (DDTM) between the sensors; and representing it in a compact way by regressing the distance-dependent coefficients as distance-dependent functions. The DDTM is measured by calculating a projective transformation matrix for varying distances between the sensors and possible targets. To do so we designed a unique experimental setup including unique Artificial Control Points (ACPs) and their detection algorithms for the two sensors. We demonstrate the utility of our approach using different experiments and evaluation criteria. PMID:26308000

Terahertz imaging using photomixers based on quantum well photodetectors

NASA Astrophysics Data System (ADS)

Zhou, T.; Li, H.; Wan, W. J.; Fu, Z. L.; Cao, J. C.

2017-10-01

Due to the fast intersubband transitions, the terahertz (THz) quantum well photodetector (QWP) is supposed to work fast. Recently it has been demonstrated that the THz QWP can detect the THz light modulated at 6.2 GHz and therefore it can be used as a photomixer [H. Li et al., Sci. Rep. 7, 3452 (2017)]. In this work, the authors report a novel active THz imaging using THz QWP photomixers. The THz radiation source used for this imaging application is a multi-mode THz quantum cascade laser (QCL) operating in continuous wave mode. When the fast THz QWP is illuminated by the multi-mode THz radiation, the intermediate frequency signal that is resulted from the frequency beating between the neighbouring THz modes of the QCL can be extracted from the QWP mesa for imaging applications. Employing the technique, the frequency can be down-converted from the THz range to the microwave regime. And therefore, the signal can then be amplified, filtered, and detected using the mature microwave technology.

MEG-BIDS, the brain imaging data structure extended to magnetoencephalography

PubMed Central

Niso, Guiomar; Gorgolewski, Krzysztof J.; Bock, Elizabeth; Brooks, Teon L.; Flandin, Guillaume; Gramfort, Alexandre; Henson, Richard N.; Jas, Mainak; Litvak, Vladimir; T. Moreau, Jeremy; Oostenveld, Robert; Schoffelen, Jan-Mathijs; Tadel, Francois; Wexler, Joseph; Baillet, Sylvain

2018-01-01

We present a significant extension of the Brain Imaging Data Structure (BIDS) to support the specific aspects of magnetoencephalography (MEG) data. MEG measures brain activity with millisecond temporal resolution and unique source imaging capabilities. So far, BIDS was a solution to organise magnetic resonance imaging (MRI) data. The nature and acquisition parameters of MRI and MEG data are strongly dissimilar. Although there is no standard data format for MEG, we propose MEG-BIDS as a principled solution to store, organise, process and share the multidimensional data volumes produced by the modality. The standard also includes well-defined metadata, to facilitate future data harmonisation and sharing efforts. This responds to unmet needs from the multimodal neuroimaging community and paves the way to further integration of other techniques in electrophysiology. MEG-BIDS builds on MRI-BIDS, extending BIDS to a multimodal data structure. We feature several data-analytics software that have adopted MEG-BIDS, and a diverse sample of open MEG-BIDS data resources available to everyone. PMID:29917016

MEG-BIDS, the brain imaging data structure extended to magnetoencephalography.

PubMed

Niso, Guiomar; Gorgolewski, Krzysztof J; Bock, Elizabeth; Brooks, Teon L; Flandin, Guillaume; Gramfort, Alexandre; Henson, Richard N; Jas, Mainak; Litvak, Vladimir; T Moreau, Jeremy; Oostenveld, Robert; Schoffelen, Jan-Mathijs; Tadel, Francois; Wexler, Joseph; Baillet, Sylvain

2018-06-19

We present a significant extension of the Brain Imaging Data Structure (BIDS) to support the specific aspects of magnetoencephalography (MEG) data. MEG measures brain activity with millisecond temporal resolution and unique source imaging capabilities. So far, BIDS was a solution to organise magnetic resonance imaging (MRI) data. The nature and acquisition parameters of MRI and MEG data are strongly dissimilar. Although there is no standard data format for MEG, we propose MEG-BIDS as a principled solution to store, organise, process and share the multidimensional data volumes produced by the modality. The standard also includes well-defined metadata, to facilitate future data harmonisation and sharing efforts. This responds to unmet needs from the multimodal neuroimaging community and paves the way to further integration of other techniques in electrophysiology. MEG-BIDS builds on MRI-BIDS, extending BIDS to a multimodal data structure. We feature several data-analytics software that have adopted MEG-BIDS, and a diverse sample of open MEG-BIDS data resources available to everyone.

Development of magneto-plasmonic nanoparticles for multimodal image-guided therapy to the brain.

PubMed

Tomitaka, Asahi; Arami, Hamed; Raymond, Andrea; Yndart, Adriana; Kaushik, Ajeet; Jayant, Rahul Dev; Takemura, Yasushi; Cai, Yong; Toborek, Michal; Nair, Madhavan

2017-01-05

Magneto-plasmonic nanoparticles are one of the emerging multi-functional materials in the field of nanomedicine. Their potential for targeting and multi-modal imaging is highly attractive. In this study, magnetic core/gold shell (MNP@Au) magneto-plasmonic nanoparticles were synthesized by citrate reduction of Au ions on magnetic nanoparticle seeds. Hydrodynamic size and optical properties of magneto-plasmonic nanoparticles synthesized with the variation of Au ions and reducing agent concentrations were evaluated. The synthesized magneto-plasmonic nanoparticles exhibited superparamagnetic properties, and their magnetic properties contributed to the concentration-dependent contrast in magnetic resonance imaging (MRI). The imaging contrast from the gold shell part of the magneto-plasmonic nanoparticles was also confirmed by X-ray computed tomography (CT). The transmigration study of the magneto-plasmonic nanoparticles using an in vitro blood-brain barrier (BBB) model proved enhanced transmigration efficiency without disrupting the integrity of the BBB, and showed potential to be used for brain diseases and neurological disorders.

Large-scale Cross-modality Search via Collective Matrix Factorization Hashing.

PubMed

Ding, Guiguang; Guo, Yuchen; Zhou, Jile; Gao, Yue

2016-09-08

By transforming data into binary representation, i.e., Hashing, we can perform high-speed search with low storage cost, and thus Hashing has collected increasing research interest in the recent years. Recently, how to generate Hashcode for multimodal data (e.g., images with textual tags, documents with photos, etc) for large-scale cross-modality search (e.g., searching semantically related images in database for a document query) is an important research issue because of the fast growth of multimodal data in the Web. To address this issue, a novel framework for multimodal Hashing is proposed, termed as Collective Matrix Factorization Hashing (CMFH). The key idea of CMFH is to learn unified Hashcodes for different modalities of one multimodal instance in the shared latent semantic space in which different modalities can be effectively connected. Therefore, accurate cross-modality search is supported. Based on the general framework, we extend it in the unsupervised scenario where it tries to preserve the Euclidean structure, and in the supervised scenario where it fully exploits the label information of data. The corresponding theoretical analysis and the optimization algorithms are given. We conducted comprehensive experiments on three benchmark datasets for cross-modality search. The experimental results demonstrate that CMFH can significantly outperform several state-of-the-art cross-modality Hashing methods, which validates the effectiveness of the proposed CMFH.

Development and Application of Multifunctional Lanthanide-Doped Nanoparticles in Medical Imaging

NASA Astrophysics Data System (ADS)

Pedraza, Francisco J., III

Medical imaging has become one of the most important tools of modern medicine soon after it was developed. Presently, several imaging modalities are available to clinicians for the detection of skeletal fractures and functional abnormalities of organs and tissues; and also an excellent tool during surgical procedures. Unfortunately, each imaging technique possesses its own strengths and inherent limitations which can be mitigated via the use of multiple imaging modalities and imaging probes. Through the use of multiple imaging modalities, it is possible to gather complementary information for a more reliable diagnosis. Each imaging technique requires its own imaging probes, providing selectivity and improved contrast. However, conventional contrast agents are incapable of providing what the new generation of multifunctional nanomaterials offer. In addition to improved selectivity and contrast, multifunctional materials possess therapeutic capabilities such as photo-thermal therapy and controlled drug delivery. Lanthanide-based nanomaterials are viable candidates for multimodal imaging agents due to possessing multifunctional capabilities, optical and chemical stability, and an intense tunable emission. This doctoral dissertation will delve into the development of lanthanide-based nanoparticles by proposing a novel multifunctional contrast agent for Near Infrared Fluorescence Imaging and Magnetic Resonance Imaging. Furthermore, the study of surface modification effects on upconversion emission and nanoparticle-cell interactions was performed. Results presented will confirm the potential application of multifunctional lanthanide-based nanomaterials as multimodal imaging probes.

Biased Competition in Visual Processing Hierarchies: A Learning Approach Using Multiple Cues.

PubMed

Gepperth, Alexander R T; Rebhan, Sven; Hasler, Stephan; Fritsch, Jannik

2011-03-01

In this contribution, we present a large-scale hierarchical system for object detection fusing bottom-up (signal-driven) processing results with top-down (model or task-driven) attentional modulation. Specifically, we focus on the question of how the autonomous learning of invariant models can be embedded into a performing system and how such models can be used to define object-specific attentional modulation signals. Our system implements bi-directional data flow in a processing hierarchy. The bottom-up data flow proceeds from a preprocessing level to the hypothesis level where object hypotheses created by exhaustive object detection algorithms are represented in a roughly retinotopic way. A competitive selection mechanism is used to determine the most confident hypotheses, which are used on the system level to train multimodal models that link object identity to invariant hypothesis properties. The top-down data flow originates at the system level, where the trained multimodal models are used to obtain space- and feature-based attentional modulation signals, providing biases for the competitive selection process at the hypothesis level. This results in object-specific hypothesis facilitation/suppression in certain image regions which we show to be applicable to different object detection mechanisms. In order to demonstrate the benefits of this approach, we apply the system to the detection of cars in a variety of challenging traffic videos. Evaluating our approach on a publicly available dataset containing approximately 3,500 annotated video images from more than 1 h of driving, we can show strong increases in performance and generalization when compared to object detection in isolation. Furthermore, we compare our results to a late hypothesis rejection approach, showing that early coupling of top-down and bottom-up information is a favorable approach especially when processing resources are constrained.

Influence of mode-beating pulse on laser-induced plasma

NASA Astrophysics Data System (ADS)

Nishihara, M.; Freund, J. B.; Glumac, N. G.; Elliott, G. S.

2018-04-01

This paper addresses the influence of mode-beating pulse on laser-induced plasma. The second harmonic of a Nd:YAG laser, operated either with the single mode or multimode, was used for non-resonant optical breakdown, and subsequent plasma development was visualized using a streak imaging system. The single mode lasing leads to a stable breakdown location and smooth envelopment of the plasma boundary, while the multimode lasing, with the dominant mode-beating frequency of 500-800 MHz, leads to fluctuations in the breakdown location, a globally modulated plasma surface, and growth of local microstructures at the plasma boundary. The distribution of the local inhomogeneity was measured from the elastic scattering signals on the streak image. The distance between the local structures agreed with the expected wavelength of hydrodynamic instability development due to the interference between the surface excited wave and transmitted wave. A numerical simulation, however, indicates that the local microstructure could also be directly generated at the peaks of the higher harmonic components if the multimode pulse contains up to the eighth harmonic of the fundamental cavity mode.

Nanoelectronics enabled chronic multimodal neural platform in a mouse ischemic model.

PubMed

Luan, Lan; Sullender, Colin T; Li, Xue; Zhao, Zhengtuo; Zhu, Hanlin; Wei, Xiaoling; Xie, Chong; Dunn, Andrew K

2018-02-01

Despite significant advancements of optical imaging techniques for mapping hemodynamics in small animal models, it remains challenging to combine imaging with spatially resolved electrical recording of individual neurons especially for longitudinal studies. This is largely due to the strong invasiveness to the living brain from the penetrating electrodes and their limited compatibility with longitudinal imaging. We implant arrays of ultraflexible nanoelectronic threads (NETs) in mice for neural recording both at the brain surface and intracortically, which maintain great tissue compatibility chronically. By mounting a cranial window atop of the NET arrays that allows for chronic optical access, we establish a multimodal platform that combines spatially resolved electrical recording of neural activity and laser speckle contrast imaging (LSCI) of cerebral blood flow (CBF) for longitudinal studies. We induce peri-infarct depolarizations (PIDs) by targeted photothrombosis, and show the ability to detect its occurrence and propagation through spatiotemporal variations in both extracellular potentials and CBF. We also demonstrate chronic tracking of single-unit neural activity and CBF over days after photothrombosis, from which we observe reperfusion and increased firing rates. This multimodal platform enables simultaneous mapping of neural activity and hemodynamic parameters at the microscale for quantitative, longitudinal comparisons with minimal perturbation to the baseline neurophysiology. The ability to spatiotemporally resolve and chronically track CBF and neural electrical activity in the same living brain region has broad applications for studying the interplay between neural and hemodynamic responses in health and in cerebrovascular and neurological pathologies. Copyright © 2017 Elsevier B.V. All rights reserved.

Validation of a deformable MRI to CT registration algorithm employing same day planning MRI for surrogate analysis.

PubMed

Padgett, Kyle R; Stoyanova, Radka; Pirozzi, Sara; Johnson, Perry; Piper, Jon; Dogan, Nesrin; Pollack, Alan

2018-03-01

Validating deformable multimodality image registrations is challenging due to intrinsic differences in signal characteristics and their spatial intensity distributions. Evaluating multimodality registrations using these spatial intensity distributions is also complicated by the fact that these metrics are often employed in the registration optimization process. This work evaluates rigid and deformable image registrations of the prostate in between diagnostic-MRI and radiation treatment planning-CT by utilizing a planning-MRI after fiducial marker placement as a surrogate. The surrogate allows for the direct quantitative analysis that can be difficult in the multimodality domain. For thirteen prostate patients, T2 images were acquired at two different time points, the first several weeks prior to planning (diagnostic-MRI) and the second on the same day as the planning-CT (planning-MRI). The diagnostic-MRI was deformed to the planning-CT utilizing a commercially available algorithm which synthesizes a deformable image registration (DIR) algorithm from local rigid registrations. The planning-MRI provided an independent surrogate for the planning-CT for assessing registration accuracy using image similarity metrics, including Pearson correlation and normalized mutual information (NMI). A local analysis was performed by looking only within the prostate, proximal seminal vesicles, penile bulb, and combined areas. The planning-MRI provided an excellent surrogate for the planning-CT with residual error in fiducial alignment between the two datasets being submillimeter, 0.78 mm. DIR was superior to the rigid registration in 11 of 13 cases demonstrating a 27.37% improvement in NMI (P < 0.009) within a regional area surrounding the prostate and associated critical organs. Pearson correlations showed similar results, demonstrating a 13.02% improvement (P < 0.013). By utilizing the planning-MRI as a surrogate for the planning-CT, an independent evaluation of registration accuracy is possible. This population provides an ideal testing ground for MRI to CT DIR by obviating the need for multimodality comparisons which are inherently more challenging. © 2018 The Authors. Journal of Applied Clinical Medical Physics published by Wiley Periodicals, Inc. on behalf of American Association of Physicists in Medicine.

A novel multimodal optical imaging system for early detection of oral cancer

PubMed Central

Malik, Bilal H.; Jabbour, Joey M.; Cheng, Shuna; Cuenca, Rodrigo; Cheng, Yi-Shing Lisa; Wright, John M.; Jo, Javier A.; Maitland, Kristen C.

2015-01-01

Objectives Several imaging techniques have been advocated as clinical adjuncts to improve identification of suspicious oral lesions. However, these have not yet shown superior sensitivity or specificity over conventional oral examination techniques. We developed a multimodal, multi-scale optical imaging system that combines macroscopic biochemical imaging of fluorescence lifetime imaging (FLIM) with subcellular morphologic imaging of reflectance confocal microscopy (RCM) for early detection of oral cancer. We tested our system on excised human oral tissues. Study Design A total of four tissue specimen were imaged. These specimens were diagnosed as one each: clinically normal, oral lichen planus, gingival hyperplasia, and superficially-invasive squamous cell carcinoma (SCC). The optical and fluorescence lifetime properties of each specimen were recorded. Results Both quantitative and qualitative differences between normal, benign and SCC lesions can be resolved with FLIM-RCM imaging. The results demonstrate that an integrated approach based on these two methods can potentially enable rapid screening and evaluation of large areas of oral epithelial tissue. Conclusions Early results from ongoing studies of imaging human oral cavity illustrate the synergistic combination of the two modalities. An adjunct device based on such optical characterization of oral mucosa can potentially be used to detect oral carcinogenesis in early stages. PMID:26725720

Fluorophore-labeling of core-crosslinked polymeric micelles for multimodal in vivo and ex vivo optical imaging

PubMed Central

Shi, Yang; Kunjachan, Sijumon; Wu, Zhuojun; Gremse, Felix; Moeckel, Diana; van Zandvoort, Marc; Kiessling, Fabian; Storm, Gert; van Nostrum, Cornelus F.; Hennink, Wim E.; Lammers, Twan

2015-01-01

Aim To enable multimodal in vivo and ex vivo optical imaging of the biodistribution and tumor accumulation of core-crosslinked polymeric micelles (CCPM). Materials & Methods mPEG-b-p(HPMAm-Lac)-based polymeric micelles, core-crosslinked via cystamine and covalently labeled with two fluorophores (Dy-676/488) were synthesized. The CCPM were intravenously injected in CT26 tumor-bearing mice. Results Upon intravenous injection, the CCPM accumulated in CT26 tumors reasonably efficiently, with values reaching ~4 %ID at 24 hours. Ex vivo TPLSM confirmed efficient extravasation of the iCCPM out of tumor blood vessels and deep penetration into the tumor interstitium. Conclusions CCPM were labeled with multiple fluorophores, and they exemplify that combining different in vivo and ex vivo optical imaging techniques is highly useful for analyzing the biodistribution and tumor accumulation of nanomedicines. PMID:25929568

Tumor Lysing Genetically Engineered T Cells Loaded with Multi-Modal Imaging Agents

NASA Astrophysics Data System (ADS)

Bhatnagar, Parijat; Alauddin, Mian; Bankson, James A.; Kirui, Dickson; Seifi, Payam; Huls, Helen; Lee, Dean A.; Babakhani, Aydin; Ferrari, Mauro; Li, King C.; Cooper, Laurence J. N.

2014-03-01

Genetically-modified T cells expressing chimeric antigen receptors (CAR) exert anti-tumor effect by identifying tumor-associated antigen (TAA), independent of major histocompatibility complex. For maximal efficacy and safety of adoptively transferred cells, imaging their biodistribution is critical. This will determine if cells home to the tumor and assist in moderating cell dose. Here, T cells are modified to express CAR. An efficient, non-toxic process with potential for cGMP compliance is developed for loading high cell number with multi-modal (PET-MRI) contrast agents (Super Paramagnetic Iron Oxide Nanoparticles - Copper-64; SPION-64Cu). This can now be potentially used for 64Cu-based whole-body PET to detect T cell accumulation region with high-sensitivity, followed by SPION-based MRI of these regions for high-resolution anatomically correlated images of T cells. CD19-specific-CAR+SPIONpos T cells effectively target in vitro CD19+ lymphoma.

Nanodiamond Landmarks for Subcellular Multimodal Optical and Electron Imaging

PubMed Central

Zurbuchen, Mark A.; Lake, Michael P.; Kohan, Sirus A.; Leung, Belinda; Bouchard, Louis-S.

2013-01-01

There is a growing need for biolabels that can be used in both optical and electron microscopies, are non-cytotoxic, and do not photobleach. Such biolabels could enable targeted nanoscale imaging of sub-cellular structures, and help to establish correlations between conjugation-delivered biomolecules and function. Here we demonstrate a sub-cellular multi-modal imaging methodology that enables localization of inert particulate probes, consisting of nanodiamonds having fluorescent nitrogen-vacancy centers. These are functionalized to target specific structures, and are observable by both optical and electron microscopies. Nanodiamonds targeted to the nuclear pore complex are rapidly localized in electron-microscopy diffraction mode to enable “zooming-in” to regions of interest for detailed structural investigations. Optical microscopies reveal nanodiamonds for in-vitro tracking or uptake-confirmation. The approach is general, works down to the single nanodiamond level, and can leverage the unique capabilities of nanodiamonds, such as biocompatibility, sensitive magnetometry, and gene and drug delivery. PMID:24036840

Multimodal image analysis of clinical influences on preterm brain development

PubMed Central

Ball, Gareth; Aljabar, Paul; Nongena, Phumza; Kennea, Nigel; Gonzalez‐Cinca, Nuria; Falconer, Shona; Chew, Andrew T.M.; Harper, Nicholas; Wurie, Julia; Rutherford, Mary A.; Edwards, A. David

2017-01-01

Objective Premature birth is associated with numerous complex abnormalities of white and gray matter and a high incidence of long‐term neurocognitive impairment. An integrated understanding of these abnormalities and their association with clinical events is lacking. The aim of this study was to identify specific patterns of abnormal cerebral development and their antenatal and postnatal antecedents. Methods In a prospective cohort of 449 infants (226 male), we performed a multivariate and data‐driven analysis combining multiple imaging modalities. Using canonical correlation analysis, we sought separable multimodal imaging markers associated with specific clinical and environmental factors and correlated to neurodevelopmental outcome at 2 years. Results We found five independent patterns of neuroanatomical variation that related to clinical factors including age, prematurity, sex, intrauterine complications, and postnatal adversity. We also confirmed the association between imaging markers of neuroanatomical abnormality and poor cognitive and motor outcomes at 2 years. Interpretation This data‐driven approach defined novel and clinically relevant imaging markers of cerebral maldevelopment, which offer new insights into the nature of preterm brain injury. Ann Neurol 2017;82:233–246 PMID:28719076

Single photon emission computed tomography/positron emission tomography imaging and targeted radionuclide therapy of melanoma: new multimodal fluorinated and iodinated radiotracers.

PubMed

Maisonial, Aurélie; Kuhnast, Bertrand; Papon, Janine; Boisgard, Raphaël; Bayle, Martine; Vidal, Aurélien; Auzeloux, Philippe; Rbah, Latifa; Bonnet-Duquennoy, Mathilde; Miot-Noirault, Elisabeth; Galmier, Marie-Josèphe; Borel, Michèle; Askienazy, Serge; Dollé, Frédéric; Tavitian, Bertrand; Madelmont, Jean-Claude; Moins, Nicole; Chezal, Jean-Michel

2011-04-28

This study reports a series of 14 new iodinated and fluorinated compounds offering both early imaging ((123)I, (124)I, (18)F) and systemic treatment ((131)I) of melanoma potentialities. The biodistribution of each (125)I-labeled tracer was evaluated in a model of melanoma B16F0-bearing mice, using in vivo serial γ scintigraphic imaging. Among this series, [(125)I]56 emerged as the most promising compound in terms of specific tumoral uptake and in vivo kinetic profile. To validate our multimodality concept, the radiosynthesis of [(18)F]56 was then optimized and this radiotracer has been successfully investigated for in vivo PET imaging of melanoma in B16F0- and B16F10-bearing mouse model. The therapeutic efficacy of [(131)I]56 was then evaluated in mice bearing subcutaneous B16F0 melanoma, and a significant slow down in tumoral growth was demonstrated. These data support further development of 56 for PET imaging ((18)F, (124)I) and targeted radionuclide therapy ((131)I) of melanoma using a single chemical structure.

Bubble-generating nano-lipid carriers for ultrasound/CT imaging-guided efficient tumor therapy.

PubMed

Zhang, Nan; Li, Jia; Hou, Ruirui; Zhang, Jiangnan; Wang, Pei; Liu, Xinyang; Zhang, Zhenzhong

2017-12-20

Ideal therapeutic effectiveness of chemotherapy is obtained only when tumor cells are exposed to a maximal drug concentration, which is often hindered by dose-limiting toxicity. We designed a bubble-generating liposomal delivery system by introducing ammonium bicarbonate and gold nanorods into folic acid-conjugated liposomes to allow both multimodal imaging and the local release of drug (doxorubicin) with hyperthermia. The key component, ammonium bicarbonate, allows a controlled, rapid release of doxorubicin to provide an effective drug concentration in the tumor microenvironment. An in vitro temperature-triggered drug release study showed that cumulative release improved more than two-fold. In addition, in vitro and in vivo studies indicated that local heat treatment or ultrasonic cavitation enhanced the therapeutic efficiency greatly. The delivery system could also serve as an excellent contrast agent to allow ultrasonic imaging and computerized tomography imaging simultaneously to further achieve the aim of accurate diagnostics. Results of this study showed that this versatile bubble-generating liposome is a promising system to provide optimal therapeutic effects that are guided by multimodal imaging. Copyright © 2017 Elsevier B.V. All rights reserved.

Separating Bulk and Surface Contributions to Electronic Excited-State Processes in Hybrid Mixed Perovskite Thin Films via Multimodal All-Optical Imaging

DOE PAGES

Simpson, Mary Jane; Doughty, Benjamin; Das, Sanjib; ...

2017-07-04

A comprehensive understanding of electronic excited-state phenomena underlying the impressive performance of solution-processed hybrid halide perovskite solar cells requires access to both spatially resolved electronic processes and corresponding sample morphological characteristics. In this paper, we demonstrate an all-optical multimodal imaging approach that enables us to obtain both electronic excited-state and morphological information on a single optical microscope platform with simultaneous high temporal and spatial resolution. Specifically, images were acquired for the same region of interest in thin films of chloride containing mixed lead halide perovskites (CH 3NH 3PbI 3–xCl x) using femtosecond transient absorption, time-integrated photoluminescence, confocal reflectance, and transmissionmore » microscopies. Comprehensive image analysis revealed the presence of surface- and bulk-dominated contributions to the various images, which describe either spatially dependent electronic excited-state properties or morphological variations across the probed region of the thin films. Finally, these results show that PL probes effectively the species near or at the film surface.« less

Direct visualization of gastrointestinal tract with lanthanide-doped BaYbF5 upconversion nanoprobes.

PubMed

Liu, Zhen; Ju, Enguo; Liu, Jianhua; Du, Yingda; Li, Zhengqiang; Yuan, Qinghai; Ren, Jinsong; Qu, Xiaogang

2013-10-01

Nanoparticulate contrast agents have attracted a great deal of attention along with the rapid development of modern medicine. Here, a binary contrast agent based on PAA modified BaYbF5:Tm nanoparticles for direct visualization of gastrointestinal (GI) tract has been designed and developed via a one-pot solvothermal route. By taking advantages of excellent colloidal stability, low cytotoxicity, and neglectable hemolysis of these well-designed nanoparticles, their feasibility as a multi-modal contrast agent for GI tract was intensively investigated. Significant enhancement of contrast efficacy relative to clinical barium meal and iodine-based contrast agent was evaluated via X-ray imaging and CT imaging in vivo. By doping Tm(3+) ions into these nanoprobes, in vivo NIR-NIR imaging was then demonstrated. Unlike some invasive imaging modalities, non-invasive imaging strategy including X-ray imaging, CT imaging, and UCL imaging for GI tract could extremely reduce the painlessness to patients, effectively facilitate imaging procedure, as well as rationality economize diagnostic time. Critical to clinical applications, long-term toxicity of our contrast agent was additionally investigated in detail, indicating their overall safety. Based on our results, PAA-BaYbF5:Tm nanoparticles were the excellent multi-modal contrast agent to integrate X-ray imaging, CT imaging, and UCL imaging for direct visualization of GI tract with low systemic toxicity. Copyright © 2013 Elsevier Ltd. All rights reserved.

Readiness of freight transportation system at special economic zone of Lhokseumawe

NASA Astrophysics Data System (ADS)

Fithra, Herman; Sirojuzilam, Saleh, Sofyan M.; Erlina

2017-11-01

Geo-economic advantages of Lhokseumawe and Aceh Utara District lies on the geographical location of Aceh crossed by Sea Lane of Communication (Sloc), the Malacca Strait. Located at the Malacca Strait, the Special Economic Zone (Kawasan Ekonomi Khusus/ KEK) of Arun Lhokseumawe has a comparative advantage to be part of the global production network or the global value chain. This study aims to determine freight transportation system to support KEK Lhokseumawe, especially the availability of multimodal transport and multimodal infrastructure. The result shows that KEK Lhokseumawe driven by SOEs in Lhokseumawe and Aceh Utara is urgent to be realized for economic acceleration and to grow new economic growth in Aceh. Multimodal transport in KEK Lhokseumawe is also available, including Ro-Ro ships, train availability from Dewantara sub-district to Muara Batu Sub-district, various types of truck with small, medium and large capacity. The available multimodal infrastructure includes international sea ports, road network connectivity with structure pavement rating of 94.62%, and railroad tracks indicating that multimodal transportation in KEK Lhokseumawe are ready to utilize. Regulatory requirements relating to the operation of all ports in KEK Lhokseumawe as export / import gate are required and serve the loading and loading activities of Containers, and as a place of origin of goods on the east coast of Aceh.

On the effects of multimodal information integration in multitasking.

PubMed

Stock, Ann-Kathrin; Gohil, Krutika; Huster, René J; Beste, Christian

2017-07-07

There have recently been considerable advances in our understanding of the neuronal mechanisms underlying multitasking, but the role of multimodal integration for this faculty has remained rather unclear. We examined this issue by comparing different modality combinations in a multitasking (stop-change) paradigm. In-depth neurophysiological analyses of event-related potentials (ERPs) were conducted to complement the obtained behavioral data. Specifically, we applied signal decomposition using second order blind identification (SOBI) to the multi-subject ERP data and source localization. We found that both general multimodal information integration and modality-specific aspects (potentially related to task difficulty) modulate behavioral performance and associated neurophysiological correlates. Simultaneous multimodal input generally increased early attentional processing of visual stimuli (i.e. P1 and N1 amplitudes) as well as measures of cognitive effort and conflict (i.e. central P3 amplitudes). Yet, tactile-visual input caused larger impairments in multitasking than audio-visual input. General aspects of multimodal information integration modulated the activity in the premotor cortex (BA 6) as well as different visual association areas concerned with the integration of visual information with input from other modalities (BA 19, BA 21, BA 37). On top of this, differences in the specific combination of modalities also affected performance and measures of conflict/effort originating in prefrontal regions (BA 6).

Differential diagnosis of left ventricular hypertrophy: usefulness of multimodality imaging and tissue characterization with cardiac magnetic resonance.

PubMed

Izgi, Cemil; Vassiliou, Vassilis; Baksi, A John; Prasad, Sanjay K

2016-11-01

Differential diagnosis of asymmetrical left ventricular hypertrophy may be challenging, particularly in patients with history of hypertension. A middle-aged man underwent an echocardiographic examination during workup for hypertension, which unexpectedly showed significant asymmetrical septal hypertrophy and raised suspicion for hypertrophic cardiomyopathy. Cardiovascular magnetic resonance confirmed the asymmetrical hypertrophy. No myocardial late gadolinium contrast enhancement was seen. However, precontrast T1 mapping revealed a low native myocardial T1 value. This was highly suggestive of Anderson-Fabry disease, which was subsequently proved with very low alpha galactosidase enzyme levels and mutation analysis. The case illustrates clinical usefulness of multimodality imaging and the novel tissue characterization techniques for assessment of left ventricular hypertrophy. © 2016, Wiley Periodicals, Inc.

SU-E-J-97: Evaluation of Multi-Modality (CT/MR/PET) Image Registration Accuracy in Radiotherapy Planning.

PubMed

Sethi, A; Rusu, I; Surucu, M; Halama, J

2012-06-01

Evaluate accuracy of multi-modality image registration in radiotherapy planning process. A water-filled anthropomorphic head phantom containing eight 'donut-shaped' fiducial markers (3 internal + 5 external) was selected for this study. Seven image sets (3CTs, 3MRs and PET) of phantom were acquired and fused in a commercial treatment planning system. First, a narrow slice (0.75mm) baseline CT scan was acquired (CT1). Subsequently, the phantom was re-scanned with a coarse slice width = 1.5mm (CT2) and after subjecting phantom to rotation/displacement (CT3). Next, the phantom was scanned in a 1.5 Tesla MR scanner and three MR image sets (axial T1, axial T2, coronal T1) were acquired at 2mm slice width. Finally, the phantom and center of fiducials were doped with 18F and a PET scan was performed with 2mm cubic voxels. All image scans (CT/MR/PET) were fused to the baseline (CT1) data using automated mutual-information based fusion algorithm. Difference between centroids of fiducial markers in various image modalities was used to assess image registration accuracy. CT/CT image registration was superior to CT/MR and CT/PET: average CT/CT fusion error was found to be 0.64 ± 0.14 mm. Corresponding values for CT/MR and CT/PET fusion were 1.33 ± 0.71mm and 1.11 ± 0.37mm. Internal markers near the center of phantom fused better than external markers placed on the phantom surface. This was particularly true for the CT/MR and CT/PET. The inferior quality of external marker fusion indicates possible distortion effects toward the edges of MR image. Peripheral targets in the PET scan may be subject to parallax error caused by depth of interaction of photons in detectors. Current widespread use of multimodality imaging in radiotherapy planning calls for periodic quality assurance of image registration process. Such studies may help improve safety and accuracy in treatment planning. © 2012 American Association of Physicists in Medicine.

Functional-structural reorganisation of the neuronal network for auditory perception in subjects with unilateral hearing loss: Review of neuroimaging studies.

PubMed

Heggdal, Peder O Laugen; Brännström, Jonas; Aarstad, Hans Jørgen; Vassbotn, Flemming S; Specht, Karsten

2016-02-01

This paper aims to provide a review of studies using neuroimaging to measure functional-structural reorganisation of the neuronal network for auditory perception after unilateral hearing loss. A literature search was performed in PubMed. Search criterions were peer reviewed original research papers in English completed by the 11th of March 2015. Twelve studies were found to use neuroimaging in subjects with unilateral hearing loss. An additional five papers not identified by the literature search were provided by a reviewer. Thus, a total of 17 studies were included in the review. Four different neuroimaging methods were used in these studies: Functional magnetic resonance imaging (fMRI) (n = 11), diffusion tensor imaging (DTI) (n = 4), T1/T2 volumetric images (n = 2), magnetic resonance spectroscopy (MRS) (n = 1). One study utilized two imaging methods (fMRI and T1 volumetric images). Neuroimaging techniques could provide valuable information regarding the effects of unilateral hearing loss on both auditory and non-auditory performance. fMRI-studies showing a bilateral BOLD-response in patients with unilateral hearing loss have not yet been followed by DTI studies confirming their microstructural correlates. In addition, the review shows that an auditory modality-specific deficit could affect multi-modal brain regions and their connections. Copyright © 2015 Elsevier B.V. All rights reserved.

Use of stereo vision and 24-bit false-color imagery to enhance visualization of multimodal confocal images

NASA Astrophysics Data System (ADS)

Beltrame, Francesco; Diaspro, Alberto; Fato, Marco; Martin, I.; Ramoino, Paola; Sobel, Irwin E.

1995-03-01

Confocal microscopy systems can be linked to 3D data oriented devices for the interactive navigation of the operator through a 3D object space. Sometimes, such environments are named `virtual reality' or `augmented reality' systems. We consider optical confocal laser scanning microscopy images, in fluorescence with various excitations and emissions, and versus time The aim of our study has been the quantitative spatial analysis of confocal data using the false-color composition technique. Starting from three 2D confocal fluorescent images at the same slice location in a given biological specimen, a new single image representation of all three parameters has been generated by the false-color technique on a HP 9000/735 workstation, connected to the confocal microscope. The color composite result of the mapping of the three parameters is displayed using a resolution of 24 bits per pixel. The operator may independently vary the mix of each of the three components in the false-color composite via three (R, G, B) mixing sliders. Furthermore, by using the pixel data in the three fluorescent component images, a 3D space containing the density distribution of these three parameters has been constructed. The histogram has been displayed in stereo: it can be used for clustering purposes from the operator, through an original thresholding algorithm.