MIRA: mutual information-based reporter algorithm for metabolic networks
Cicek, A. Ercument; Roeder, Kathryn; Ozsoyoglu, Gultekin
2014-01-01
Motivation: Discovering the transcriptional regulatory architecture of the metabolism has been an important topic to understand the implications of transcriptional fluctuations on metabolism. The reporter algorithm (RA) was proposed to determine the hot spots in metabolic networks, around which transcriptional regulation is focused owing to a disease or a genetic perturbation. Using a z-score-based scoring scheme, RA calculates the average statistical change in the expression levels of genes that are neighbors to a target metabolite in the metabolic network. The RA approach has been used in numerous studies to analyze cellular responses to the downstream genetic changes. In this article, we propose a mutual information-based multivariate reporter algorithm (MIRA) with the goal of eliminating the following problems in detecting reporter metabolites: (i) conventional statistical methods suffer from small sample sizes, (ii) as z-score ranges from minus to plus infinity, calculating average scores can lead to canceling out opposite effects and (iii) analyzing genes one by one, then aggregating results can lead to information loss. MIRA is a multivariate and combinatorial algorithm that calculates the aggregate transcriptional response around a metabolite using mutual information. We show that MIRA’s results are biologically sound, empirically significant and more reliable than RA. Results: We apply MIRA to gene expression analysis of six knockout strains of Escherichia coli and show that MIRA captures the underlying metabolic dynamics of the switch from aerobic to anaerobic respiration. We also apply MIRA to an Autism Spectrum Disorder gene expression dataset. Results indicate that MIRA reports metabolites that highly overlap with recently found metabolic biomarkers in the autism literature. Overall, MIRA is a promising algorithm for detecting metabolic drug targets and understanding the relation between gene expression and metabolic activity. Availability and
Mutual information image registration based on improved bee evolutionary genetic algorithm
NASA Astrophysics Data System (ADS)
Xu, Gang; Tu, Jingzhi
2009-07-01
In recent years, the mutual information is regarded as a more efficient similarity metrics in the image registration. According to the features of mutual information image registration, the Bee Evolution Genetic Algorithm (BEGA) is chosen for optimizing parameters, which imitates swarm mating. Besides, we try our best adaptively set the initial parameters to improve the BEGA. The programming result shows the wonderful precision of the algorithm.
IR and visual image registration based on mutual information and PSO-Powell algorithm
NASA Astrophysics Data System (ADS)
Zhuang, Youwen; Gao, Kun; Miu, Xianghu
2014-11-01
Infrared and visual image registration has a wide application in the fields of remote sensing and military. Mutual information (MI) has proved effective and successful in infrared and visual image registration process. To find the most appropriate registration parameters, optimal algorithms, such as Particle Swarm Optimization (PSO) algorithm or Powell search method, are often used. The PSO algorithm has strong global search ability and search speed is fast at the beginning, while the weakness is low search performance in late search stage. In image registration process, it often takes a lot of time to do useless search and solution's precision is low. Powell search method has strong local search ability. However, the search performance and time is more sensitive to initial values. In image registration, it is often obstructed by local maximum and gets wrong results. In this paper, a novel hybrid algorithm, which combined PSO algorithm and Powell search method, is proposed. It combines both advantages that avoiding obstruction caused by local maximum and having higher precision. Firstly, using PSO algorithm gets a registration parameter which is close to global minimum. Based on the result in last stage, the Powell search method is used to find more precision registration parameter. The experimental result shows that the algorithm can effectively correct the scale, rotation and translation additional optimal algorithm. It can be a good solution to register infrared difference of two images and has a greater performance on time and precision than traditional and visible images.
NASA Astrophysics Data System (ADS)
Zhang, Hong; Sun, Yanfeng; Zhai, Bing; Wang, Yiding
2013-07-01
This paper studies on the image registration of the medical images. Wavelet transform is adopted to decompose the medical images because the resolution of the medical image is high and the computational amount of the registration is large. Firstly, the low frequency sub-images are matched. Then source images are matched. The image registration was fulfilled by the ant colony optimization algorithm to search the extremum of the mutual information. The experiment result demonstrates the proposed approach can not only reduce calculation amount, but also skip from the local extremum during optimization process, and search the optimization value.
Research on non rigid registration algorithm of DCE-MRI based on mutual information and optical flow
NASA Astrophysics Data System (ADS)
Yu, Shihua; Wang, Rui; Wang, Kaiyu; Xi, Mengmeng; Zheng, Jiashuo; Liu, Hui
2015-07-01
Image matching plays a very important role in the field of medical image, while the two image registration methods based on the mutual information and the optical flow are very effective. The experimental results show that the two methods have their prominent advantages. The method based on mutual information is good for the overall displacement, while the method based on optical flow is very sensitive to small deformation. In the breast DCE-MRI images studied in this paper, there is not only overall deformation caused by the patient, but also non rigid small deformation caused by respiratory deformation. In view of the above situation, the single-image registration algorithms cannot meet the actual needs of complex situations. After a comprehensive analysis to the advantages and disadvantages of these two methods, this paper proposes a registration algorithm of combining mutual information with optical flow field, and applies subtraction images of the reference image and the floating image as the main criterion to evaluate the registration effect, at the same time, applies the mutual information between image sequence values as auxiliary criterion. With the test of the example, this algorithm has obtained a better accuracy and reliability in breast DCE-MRI image sequences.
Hierarchical clustering using mutual information
NASA Astrophysics Data System (ADS)
Kraskov, A.; Stögbauer, H.; Andrzejak, R. G.; Grassberger, P.
2005-04-01
We present a conceptually simple method for hierarchical clustering of data called mutual information clustering (MIC) algorithm. It uses mutual information (MI) as a similarity measure and exploits its grouping property: The MI between three objects X, Y, and Z is equal to the sum of the MI between X and Y, plus the MI between Z and the combined object (XY). We use this both in the Shannon (probabilistic) version of information theory and in the Kolmogorov (algorithmic) version. We apply our method to the construction of phylogenetic trees from mitochondrial DNA sequences and to the output of independent components analysis (ICA) as illustrated with the ECG of a pregnant woman.
Estimating mutual information.
Kraskov, Alexander; Stögbauer, Harald; Grassberger, Peter
2004-06-01
We present two classes of improved estimators for mutual information M(X,Y), from samples of random points distributed according to some joint probability density mu(x,y). In contrast to conventional estimators based on binnings, they are based on entropy estimates from k -nearest neighbor distances. This means that they are data efficient (with k=1 we resolve structures down to the smallest possible scales), adaptive (the resolution is higher where data are more numerous), and have minimal bias. Indeed, the bias of the underlying entropy estimates is mainly due to nonuniformity of the density at the smallest resolved scale, giving typically systematic errors which scale as functions of k/N for N points. Numerically, we find that both families become exact for independent distributions, i.e. the estimator M(X,Y) vanishes (up to statistical fluctuations) if mu(x,y)=mu(x)mu(y). This holds for all tested marginal distributions and for all dimensions of x and y. In addition, we give estimators for redundancies between more than two random variables. We compare our algorithms in detail with existing algorithms. Finally, we demonstrate the usefulness of our estimators for assessing the actual independence of components obtained from independent component analysis (ICA), for improving ICA, and for estimating the reliability of blind source separation. PMID:15244698
Mutual information in classical spin models
NASA Astrophysics Data System (ADS)
Wilms, Johannes; Troyer, Matthias; Verstraete, Frank
2011-10-01
The total many-body correlations present in finite temperature classical spin systems are studied using the concept of mutual information. As opposed to zero-temperature quantum phase transitions, the total correlations are not maximal at the phase transition, but reach a maximum in the high-temperature paramagnetic phase. The Shannon mutual information and the Renyi mutual information in both Ising and Potts models in two dimensions are calculated numerically by combining matrix product state algorithms and Monte Carlo sampling techniques.
Quantum algorithm for SAT problem andquantum mutual entropy
NASA Astrophysics Data System (ADS)
Ohya, Masanori
2005-02-01
It is von Neumann who opened the window for today's information epoch. He definedquantum entropy including Shannon's information more than 20 years ahead of Shannon, and he explained what computation means mathematically. In this paper I discuss two problems studied recently by me and my coworkers. One of them concerns a quantum algorithm in a generalized sense solving the SAT problem (one of NP complete problems) and another concerns quantum mutual entropy properly describing quantum communication processes.
Mutual information analysis of JPEG2000 contexts
NASA Astrophysics Data System (ADS)
Liu, Zhen; Karam, Lina J.
2003-05-01
Context-based arithmetic coding has been widely adopted in image and video compression and is a key component of the new JPEG2000 image compression standard. In this paper, the contexts used in JPEG2000 are analyzed using the mutual information, which has a direct link with the compression performance. We first show that, when combining the contexts, the mutual information between the contexts and the encoded data will decrease unless the conditional probability distributions of the combined contexts are the same. Given I, the initial number of contexts, and F, the final desired number of contexts, there are S(I, F) possible context classification schemes where S(I, F) is called the Stirling number of the second kind. The optimal classification scheme is the one that gives the maximum mutual information. Instead of exhaustive search, the optimal classification scheme can be obtained through a modified Generalized Lloyd algorithm with the relative entropy as the distortion metric. For binary arithmetic coding, the search complexity can be reduced by using the dynamic programming. Our experimental results show that the JPEG2000 contexts capture very well the correlations among the wavelet coefficients. At the same time, the number of contexts used as part of the standard can be reduced without loss in the coding performance.
Generalized mutual information and Tsirelson's bound
Wakakuwa, Eyuri; Murao, Mio
2014-12-04
We introduce a generalization of the quantum mutual information between a classical system and a quantum system into the mutual information between a classical system and a system described by general probabilistic theories. We apply this generalized mutual information (GMI) to a derivation of Tsirelson's bound from information causality, and prove that Tsirelson's bound can be derived from the chain rule of the GMI. By using the GMI, we formulate the 'no-supersignalling condition' (NSS), that the assistance of correlations does not enhance the capability of classical communication. We prove that NSS is never violated in any no-signalling theory.
Mutual information rate and bounds for it.
Baptista, Murilo S; Rubinger, Rero M; Viana, Emilson R; Sartorelli, José C; Parlitz, Ulrich; Grebogi, Celso
2012-01-01
The amount of information exchanged per unit of time between two nodes in a dynamical network or between two data sets is a powerful concept for analysing complex systems. This quantity, known as the mutual information rate (MIR), is calculated from the mutual information, which is rigorously defined only for random systems. Moreover, the definition of mutual information is based on probabilities of significant events. This work offers a simple alternative way to calculate the MIR in dynamical (deterministic) networks or between two time series (not fully deterministic), and to calculate its upper and lower bounds without having to calculate probabilities, but rather in terms of well known and well defined quantities in dynamical systems. As possible applications of our bounds, we study the relationship between synchronisation and the exchange of information in a system of two coupled maps and in experimental networks of coupled oscillators. PMID:23112809
Mutual information and spontaneous symmetry breaking
NASA Astrophysics Data System (ADS)
Hamma, A.; Giampaolo, S. M.; Illuminati, F.
2016-01-01
We show that the metastable, symmetry-breaking ground states of quantum many-body Hamiltonians have vanishing quantum mutual information between macroscopically separated regions and are thus the most classical ones among all possible quantum ground states. This statement is obvious only when the symmetry-breaking ground states are simple product states, e.g., at the factorization point. On the other hand, symmetry-breaking states are in general entangled along the entire ordered phase, and to show that they actually feature the least macroscopic correlations compared to their symmetric superpositions is highly nontrivial. We prove this result in general, by considering the quantum mutual information based on the two-Rényi entanglement entropy and using a locality result stemming from quasiadiabatic continuation. Moreover, in the paradigmatic case of the exactly solvable one-dimensional quantum X Y model, we further verify the general result by considering also the quantum mutual information based on the von Neumann entanglement entropy.
Distribution of Mutual Information in Multipartite States
NASA Astrophysics Data System (ADS)
Maziero, Jonas
2014-06-01
Using the relative entropy of total correlation, we derive an expression relating the mutual information of n-partite pure states to the sum of the mutual informations and entropies of its marginals and analyze some of its implications. Besides, by utilizing the extended strong subadditivity of von Neumann entropy, we obtain generalized monogamy relations for the total correlation in three-partite mixed states. These inequalities lead to a tight lower bound for this correlation in terms of the sum of the bipartite mutual informations. We use this bound to propose a measure for residual three-partite total correlation and discuss the non-applicability of this kind of quantifier to measure genuine multiparty correlations.
Efficient algorithm to compute mutually connected components in interdependent networks.
Hwang, S; Choi, S; Lee, Deokjae; Kahng, B
2015-02-01
Mutually connected components (MCCs) play an important role as a measure of resilience in the study of interdependent networks. Despite their importance, an efficient algorithm to obtain the statistics of all MCCs during the removal of links has thus far been absent. Here, using a well-known fully dynamic graph algorithm, we propose an efficient algorithm to accomplish this task. We show that the time complexity of this algorithm is approximately O(N(1.2)) for random graphs, which is more efficient than O(N(2)) of the brute-force algorithm. We confirm the correctness of our algorithm by comparing the behavior of the order parameter as links are removed with existing results for three types of double-layer multiplex networks. We anticipate that this algorithm will be used for simulations of large-size systems that have been previously inaccessible. PMID:25768559
Blasting and Zipping: Sequence Alignment and Mutual Information
NASA Astrophysics Data System (ADS)
Penner, Orion; Grassberger, Peter; Paczuski, Maya
2009-03-01
Alignment of biological sequences such as DNA, RNA or proteins is one of the most widely used tools in computational bioscience. While the accomplishments of sequence alignment algorithms are undeniable the fact remains that these algorithms are based upon heuristic scoring schemes. Therefore, these algorithms do not provide model independent and objective measures for how similar two (or more) sequences actually are. Although information theory provides such a similarity measure - the mutual information (MI) - numerous previous attempts to connect sequence alignment and information have not produced realistic estimates for the MI from a given alignment. We report on a simple and flexible approach to get robust estimates of MI from global alignments. The presented results may help establish MI as a reliable tool for evaluating the quality of global alignments, judging the relative merits of different alignment algorithms, and estimating the significance of specific alignments.
Problem decomposition by mutual information and force-based clustering
NASA Astrophysics Data System (ADS)
Otero, Richard Edward
alternative global optimizer, called MIMIC, which is unrelated to Genetic Algorithms. Advancement to the current practice demonstrates the use of MIMIC as a global method that explicitly models problem structure with mutual information, providing an alternate method for globally searching multi-modal domains. By leveraging discovered problem inter- dependencies, MIMIC may be appropriate for highly coupled problems or those with large function evaluation cost. This work introduces a useful addition to the MIMIC algorithm that enables its use on continuous input variables. By leveraging automatic decision tree generation methods from Machine Learning and a set of randomly generated test problems, decision trees for which method to apply are also created, quantifying decomposition performance over a large region of the design space.
Generalized mutual information of quantum critical chains
NASA Astrophysics Data System (ADS)
Alcaraz, F. C.; Rajabpour, M. A.
2015-04-01
We study the generalized mutual information I˜n of the ground state of different critical quantum chains. The generalized mutual information definition that we use is based on the well established concept of the Rényi divergence. We calculate this quantity numerically for several distinct quantum chains having either discrete Z (Q ) symmetries (Q -state Potts model with Q =2 ,3 ,4 and Z (Q ) parafermionic models with Q =5 ,6 ,7 ,8 and also Ashkin-Teller model with different anisotropies) or the U (1 ) continuous symmetries (Klein-Gordon field theory, X X Z and spin-1 Fateev-Zamolodchikov quantum chains with different anisotropies). For the spin chains these calculations were done by expressing the ground-state wave functions in two special bases. Our results indicate some general behavior for particular ranges of values of the parameter n that defines I˜n. For a system, with total size L and subsystem sizes ℓ and L -ℓ , the I˜n has a logarithmic leading behavior given by c/˜n4 log[L/π sin(π/ℓ L ) ] where the coefficient c˜n is linearly dependent on the central charge c of the underlying conformal field theory describing the system's critical properties.
An efficient algorithm for direction finding against unknown mutual coupling.
Wang, Weijiang; Ren, Shiwei; Ding, Yingtao; Wang, Haoyu
2014-01-01
In this paper, an algorithm of direction finding is proposed in the presence of unknown mutual coupling. The preliminary direction of arrival (DOA) is estimated using the whole array for high resolution. Further refinement can then be conducted by estimating the angularly dependent coefficients (ADCs) with the subspace theory. The mutual coupling coefficients are finally determined by solving the least squares problem with all of the ADCs utilized without discarding any. Simulation results show that the proposed method can achieve better performance at a low signal-to-noise ratio (SNR) with a small-sized array and is more robust, compared with the similar processes employing the initial DOA estimation and further improvement iteratively. PMID:25347587
An Efficient Algorithm for Direction Finding against Unknown Mutual Coupling
Wang, Weijiang; Ren, Shiwei; Ding, Yingtao; Wang, Haoyu
2014-01-01
In this paper, an algorithm of direction finding is proposed in the presence of unknown mutual coupling. The preliminary direction of arrival (DOA) is estimated using the whole array for high resolution. Further refinement can then be conducted by estimating the angularly dependent coefficients (ADCs) with the subspace theory. The mutual coupling coefficients are finally determined by solving the least squares problem with all of the ADCs utilized without discarding any. Simulation results show that the proposed method can achieve better performance at a low signal-to-noise ratio (SNR) with a small-sized array and is more robust, compared with the similar processes employing the initial DOA estimation and further improvement iteratively. PMID:25347587
Graph embedded nonparametric mutual information for supervised dimensionality reduction.
Bouzas, Dimitrios; Arvanitopoulos, Nikolaos; Tefas, Anastasios
2015-05-01
In this paper, we propose a novel algorithm for dimensionality reduction that uses as a criterion the mutual information (MI) between the transformed data and their corresponding class labels. The MI is a powerful criterion that can be used as a proxy to the Bayes error rate. Furthermore, recent quadratic nonparametric implementations of MI are computationally efficient and do not require any prior assumptions about the class densities. We show that the quadratic nonparametric MI can be formulated as a kernel objective in the graph embedding framework. Moreover, we propose its linear equivalent as a novel linear dimensionality reduction algorithm. The derived methods are compared against the state-of-the-art dimensionality reduction algorithms with various classifiers and on various benchmark and real-life datasets. The experimental results show that nonparametric MI as an optimization objective for dimensionality reduction gives comparable and in most of the cases better results compared with other dimensionality reduction methods. PMID:25881367
Mutual information-based facial expression recognition
NASA Astrophysics Data System (ADS)
Hazar, Mliki; Hammami, Mohamed; Hanêne, Ben-Abdallah
2013-12-01
This paper introduces a novel low-computation discriminative regions representation for expression analysis task. The proposed approach relies on interesting studies in psychology which show that most of the descriptive and responsible regions for facial expression are located around some face parts. The contributions of this work lie in the proposition of new approach which supports automatic facial expression recognition based on automatic regions selection. The regions selection step aims to select the descriptive regions responsible or facial expression and was performed using Mutual Information (MI) technique. For facial feature extraction, we have applied Local Binary Patterns Pattern (LBP) on Gradient image to encode salient micro-patterns of facial expressions. Experimental studies have shown that using discriminative regions provide better results than using the whole face regions whilst reducing features vector dimension.
Glacier Surface Monitoring by Maximizing Mutual Information
NASA Astrophysics Data System (ADS)
Erten, E.; Rossi, C.; Hajnsek, I.
2012-07-01
The contribution of Polarimetric Synthetic Aperture Radar (PolSAR) images compared with the single-channel SAR in terms of temporal scene characterization has been found and described to add valuable information in the literature. However, despite a number of recent studies focusing on single polarized glacier monitoring, the potential of polarimetry to estimate the surface velocity of glaciers has not been explored due to the complex mechanism of polarization through glacier/snow. In this paper, a new approach to the problem of monitoring glacier surface velocity is proposed by means of temporal PolSAR images, using a basic concept from information theory: Mutual Information (MI). The proposed polarimetric tracking method applies the MI to measure the statistical dependence between temporal polarimetric images, which is assumed to be maximal if the images are geometrically aligned. Since the proposed polarimetric tracking method is very powerful and general, it can be implemented into any kind of multivariate remote sensing data such as multi-spectral optical and single-channel SAR images. The proposed polarimetric tracking is then used to retrieve surface velocity of Aletsch glacier located in Switzerland and of Inyltshik glacier in Kyrgyzstan with two different SAR sensors; Envisat C-band (single polarized) and DLR airborne L-band (fully polarimetric) systems, respectively. The effect of number of channel (polarimetry) into tracking investigations demonstrated that the presence of snow, as expected, effects the location of the phase center in different polarization, such as glacier tracking with temporal HH compared to temporal VV channels. Shortly, a change in polarimetric signature of the scatterer can change the phase center, causing a question of how much of what I am observing is motion then penetration. In this paper, it is shown that considering the multi-channel SAR statistics, it is possible to optimize the separate these contributions.
Mutual information-based LPI optimisation for radar network
NASA Astrophysics Data System (ADS)
Shi, Chenguang; Zhou, Jianjiang; Wang, Fei; Chen, Jun
2015-07-01
Radar network can offer significant performance improvement for target detection and information extraction employing spatial diversity. For a fixed number of radars, the achievable mutual information (MI) for estimating the target parameters may extend beyond a predefined threshold with full power transmission. In this paper, an effective low probability of intercept (LPI) optimisation algorithm is presented to improve LPI performance for radar network. Based on radar network system model, we first provide Schleher intercept factor for radar network as an optimisation metric for LPI performance. Then, a novel LPI optimisation algorithm is presented, where for a predefined MI threshold, Schleher intercept factor for radar network is minimised by optimising the transmission power allocation among radars in the network such that the enhanced LPI performance for radar network can be achieved. The genetic algorithm based on nonlinear programming (GA-NP) is employed to solve the resulting nonconvex and nonlinear optimisation problem. Some simulations demonstrate that the proposed algorithm is valuable and effective to improve the LPI performance for radar network.
Spatial Mutual Information Based Hyperspectral Band Selection for Classification
2015-01-01
The amount of information involved in hyperspectral imaging is large. Hyperspectral band selection is a popular method for reducing dimensionality. Several information based measures such as mutual information have been proposed to reduce information redundancy among spectral bands. Unfortunately, mutual information does not take into account the spatial dependency between adjacent pixels in images thus reducing its robustness as a similarity measure. In this paper, we propose a new band selection method based on spatial mutual information. As validation criteria, a supervised classification method using support vector machine (SVM) is used. Experimental results of the classification of hyperspectral datasets show that the proposed method can achieve more accurate results. PMID:25918742
[Human cerebral image registration using generalized mutual information].
Zhang, Jingzhou; Li, Ting; Zhang, Jia
2008-12-01
Medical image registration is a highlight of actual research on medical image processing. Based onsimilarity measure of Shannon entropy, a new generalized distance measurement based on Rényi entropy applied to image rigid registration is introduced and is called here generalized mutual information (GMI). It is used in three dimensional cerebral image registration experiments. The simulation results show that generalized distance measurement and Shannon entropy measurement apply to different areas; that the registration measure based o n generalized distance is a natural extension of mutual information of Shannon entropy. The results prove that generalized mutual information uses less time than simple mutual information does, and the new similarity measure manifests higher degree of consistency between the two cerebral registration images. Also, the registration results provide the clinical diagnoses with more important references. In conclusion, generalized mutual information has satisfied the demands of clinical application to a wide extent. PMID:19166197
Mutual information between SSH and SST fields
NASA Astrophysics Data System (ADS)
Le goff, Clément; Chapron, Bertrand; Fablet, Ronan; Tandeo, Pierre; Autret, Emmanuelle; Ailliot, Pierre
2015-04-01
Mutual information between SST and SSH Investigations to relate satellite SST and SSH measurements in the Agulhas return current region are presented. In this study, we focus on the use of SSH and SST maps obtained during the year 2004, corresponding to a particularly well-sampled period for altimetry. The SST and SSH anomalies are then obtained as high-pass filtered fields, to analyze scales smaller than approximately 300km. As revealed, we clearly distinguish different regimes. During the winter months, a marked strong correlation between fields of SSH and SST anomalies is clearly revealed. During the summer months, a much lower correlation is found. Further conditioning the analysis to separate the areas of positive and negative SSH anomalies, it is then obtained, for both summer and winter periods, that aeras of negative SSH anomalies always correspond with areas of negative SST anomalies. This high correspondance also applies in winter but only for areas of positive SSH anomalies, which indeed well match with areas of positive SST anomalies. In summer, this high correspondance is lost, and areas with positive SSH anomalies do not necessarily correspond to positive SST anomalies. Accordingly, such an effect affects and weakens the overall SST/SSH correlation during the summer months. Yet, the areas of positive SSH anomalies are not fully disconnected from the areas of positive SST anomalies. For these cases, observations and results demonstrate a systematic spatial shift between them. This suggests the influence of the mixed layer depth and wind speed to control the spatial correspondance between SST and SSH anomalies, especially below regions of positive SSH anomalies. In such cases, the upper layer SST anomalies are certainly advected by the interior flow to also provide means to relate surface observations and interior dynamics.
A new mutually reinforcing network node and link ranking algorithm
Wang, Zhenghua; Dueñas-Osorio, Leonardo; Padgett, Jamie E.
2015-01-01
This study proposes a novel Normalized Wide network Ranking algorithm (NWRank) that has the advantage of ranking nodes and links of a network simultaneously. This algorithm combines the mutual reinforcement feature of Hypertext Induced Topic Selection (HITS) and the weight normalization feature of PageRank. Relative weights are assigned to links based on the degree of the adjacent neighbors and the Betweenness Centrality instead of assigning the same weight to every link as assumed in PageRank. Numerical experiment results show that NWRank performs consistently better than HITS, PageRank, eigenvector centrality, and edge betweenness from the perspective of network connectivity and approximate network flow, which is also supported by comparisons with the expensive N-1 benchmark removal criteria based on network efficiency. Furthermore, it can avoid some problems, such as the Tightly Knit Community effect, which exists in HITS. NWRank provides a new inexpensive way to rank nodes and links of a network, which has practical applications, particularly to prioritize resource allocation for upgrade of hierarchical and distributed networks, as well as to support decision making in the design of networks, where node and link importance depend on a balance of local and global integrity. PMID:26492958
NASA Astrophysics Data System (ADS)
Cariou, Claude; Chehdi, Kacem
2013-10-01
We investigate the potential of multidimensional mutual information for the registration of multi-spectral remote sensing images. We devise a gradient flow algorithm which iteratively maximizes the multidimensional mutual information with respect to a differentiable displacement map, accounting for partial derivatives of the multivariate joint distribution and the multivariate marginal of the float image with respect to each variable of the mutual information derivative. The resulting terms are shown to weight the band specific gradients of the warp image, and we propose in addition to compute them with a method based on the k-nearest neighbours. We apply our method to the registration of Ikonos and CHRIS-Proba images over the region of Baabdat, Lebanon, for purposes of cedar pines detection. A comparison between (crossed) single band and multi-band registration results obtained shows that using the multidimensional mutual information brings a significant gain in positional accuracy and is suitable for multispectral remote sensing image registration.
Quantum Conditional Mutual Information, Reconstructed States, and State Redistribution.
Brandão, Fernando G S L; Harrow, Aram W; Oppenheim, Jonathan; Strelchuk, Sergii
2015-07-31
We give two strengthenings of an inequality for the quantum conditional mutual information of a tripartite quantum state recently proved by Fawzi and Renner, connecting it with the ability to reconstruct the state from its bipartite reductions. Namely, we show that the conditional mutual information is an upper bound on the regularized relative entropy distance between the quantum state and its reconstructed version. It is also an upper bound for the measured relative entropy distance of the state to its reconstructed version. The main ingredient of the proof is the fact that the conditional mutual information is the optimal quantum communication rate in the task of state redistribution. PMID:26274402
Nonlinear pattern analysis of ventricular premature beats by mutual information
NASA Technical Reports Server (NTRS)
Osaka, M.; Saitoh, H.; Yokoshima, T.; Kishida, H.; Hayakawa, H.; Cohen, R. J.
1997-01-01
The frequency of ventricular premature beats (VPBs) has been related to the risk of mortality. However, little is known about the temporal pattern of occurrence of VPBs and its relationship to autonomic activity. Hence, we applied a general correlation measure, mutual information, to quantify how VPBs are generated over time. We also used mutual information to determine the correlation between VPB production and heart rate in order to evaluate effects of autonomic activity on VPB production. We examined twenty subjects with more than 3000 VPBs/day and simulated random time series of VPB occurrence. We found that mutual information values could be used to characterize quantitatively the temporal patterns of VPB generation. Our data suggest that VPB production is not random and VPBs generated with a higher value of mutual information may be more greatly affected by autonomic activity.
Robust volumetric change detection using mutual information with 3D fractals
NASA Astrophysics Data System (ADS)
Rahmes, Mark; Akbari, Morris; Henning, Ronda; Pokorny, John
2014-06-01
We discuss a robust method for quantifying change of multi-temporal remote sensing point data in the presence of affine registration errors. Three dimensional image processing algorithms can be used to extract and model an electronic module, consisting of a self-contained assembly of electronic components and circuitry, using an ultrasound scanning sensor. Mutual information (MI) is an effective measure of change. We propose a multi-resolution 3D fractal algorithm which is a novel extension to MI or regional mutual information (RMI). Our method is called fractal mutual information (FMI). This extension efficiently takes neighborhood fractal patterns of corresponding voxels (3D pixels) into account. The goal of this system is to quantify the change in a module due to tampering and provide a method for quantitative and qualitative change detection and analysis.
Perotti, Juan Ignacio; Tessone, Claudio Juan; Caldarelli, Guido
2015-12-01
The quest for a quantitative characterization of community and modular structure of complex networks produced a variety of methods and algorithms to classify different networks. However, it is not clear if such methods provide consistent, robust, and meaningful results when considering hierarchies as a whole. Part of the problem is the lack of a similarity measure for the comparison of hierarchical community structures. In this work we give a contribution by introducing the hierarchical mutual information, which is a generalization of the traditional mutual information and makes it possible to compare hierarchical partitions and hierarchical community structures. The normalized version of the hierarchical mutual information should behave analogously to the traditional normalized mutual information. Here the correct behavior of the hierarchical mutual information is corroborated on an extensive battery of numerical experiments. The experiments are performed on artificial hierarchies and on the hierarchical community structure of artificial and empirical networks. Furthermore, the experiments illustrate some of the practical applications of the hierarchical mutual information, namely the comparison of different community detection methods and the study of the consistency, robustness, and temporal evolution of the hierarchical modular structure of networks. PMID:26764762
NASA Astrophysics Data System (ADS)
Perotti, Juan Ignacio; Tessone, Claudio Juan; Caldarelli, Guido
2015-12-01
The quest for a quantitative characterization of community and modular structure of complex networks produced a variety of methods and algorithms to classify different networks. However, it is not clear if such methods provide consistent, robust, and meaningful results when considering hierarchies as a whole. Part of the problem is the lack of a similarity measure for the comparison of hierarchical community structures. In this work we give a contribution by introducing the hierarchical mutual information, which is a generalization of the traditional mutual information and makes it possible to compare hierarchical partitions and hierarchical community structures. The normalized version of the hierarchical mutual information should behave analogously to the traditional normalized mutual information. Here the correct behavior of the hierarchical mutual information is corroborated on an extensive battery of numerical experiments. The experiments are performed on artificial hierarchies and on the hierarchical community structure of artificial and empirical networks. Furthermore, the experiments illustrate some of the practical applications of the hierarchical mutual information, namely the comparison of different community detection methods and the study of the consistency, robustness, and temporal evolution of the hierarchical modular structure of networks.
Model term selection for spatio-temporal system identification using mutual information
NASA Astrophysics Data System (ADS)
Wang, Shu; Wei, Hua-Liang; Coca, Daniel; Billings, Stephen A.
2013-02-01
A new mutual information based algorithm is introduced for term selection in spatio-temporal models. A generalised cross validation procedure is also introduced for model length determination and examples based on cellular automata, coupled map lattice and partial differential equations are described.
Rényi generalizations of the conditional quantum mutual information
Berta, Mario; Seshadreesan, Kaushik P.; Wilde, Mark M.
2015-02-15
The conditional quantum mutual information I(A; B|C) of a tripartite state ρ{sub ABC} is an information quantity which lies at the center of many problems in quantum information theory. Three of its main properties are that it is non-negative for any tripartite state, that it decreases under local operations applied to systems A and B, and that it obeys the duality relation I(A; B|C) = I(A; B|D) for a four-party pure state on systems ABCD. The conditional mutual information also underlies the squashed entanglement, an entanglement measure that satisfies all of the axioms desired for an entanglement measure. As such, it has been an open question to find Rényi generalizations of the conditional mutual information, that would allow for a deeper understanding of the original quantity and find applications beyond the traditional memoryless setting of quantum information theory. The present paper addresses this question, by defining different α-Rényi generalizations I{sub α}(A; B|C) of the conditional mutual information, some of which we can prove converge to the conditional mutual information in the limit α → 1. Furthermore, we prove that many of these generalizations satisfy non-negativity, duality, and monotonicity with respect to local operations on one of the systems A or B (with it being left as an open question to prove that monotonicity holds with respect to local operations on both systems). The quantities defined here should find applications in quantum information theory and perhaps even in other areas of physics, but we leave this for future work. We also state a conjecture regarding the monotonicity of the Rényi conditional mutual informations defined here with respect to the Rényi parameter α. We prove that this conjecture is true in some special cases and when α is in a neighborhood of one.
Quadratic mutual information for dimensionality reduction and classification
NASA Astrophysics Data System (ADS)
Gray, David M.; Principe, José C.
2010-04-01
A research area based on the application of information theory to machine learning has attracted considerable interest in the last few years. This research area has been coined information-theoretic learning within the community. In this paper we apply elements of information-theoretic learning to the problem of automatic target recognition (ATR). A number of researchers have previously shown the benefits of designing classifiers based on maximizing the mutual information between the class data and the class labels. Following prior research in information-theoretic learning, in the current results we show that quadratic mutual information, derived using a special case of the more general Renyi's entropy, can be used for classifier design. In this implementation, a simple subspace projection classifier is formulated to find the optimal projection weights such that the quadratic mutual information between the class data and the class labels is maximized. This subspace projection accomplishes a dimensionality reduction of the raw data set wherein information about the class membership is retained while irrelevant information is discarded. A subspace projection based on this criterion preserves as much class discriminability as possible within the subspace. For this paper, laser radar images are used to demonstrate the results. Classification performance against this data set is compared for a gradient descent MLP classifier and a quadratic mutual information MLP classifier.
Thermalization of mutual information in hyperscaling violating backgrounds
NASA Astrophysics Data System (ADS)
Tanhayi, M. Reza
2016-03-01
We study certain features of scaling behaviors of the mutual information during a process of thermalization, more precisely we extend the time scaling behavior of mutual information which has been discussed in [1] to time-dependent hyperscaling violating geometries. We use the holographic description of entanglement entropy for two disjoint system consisting of two parallel strips whose widths are much larger than the separation between them. We show that during the thermalization process, the dynamical exponent plays a crucial rule in reading the general time scaling behavior of mutual information (e.g., at the pre-local-equilibration regime). It is shown that the scaling violating parameter can be employed to define an effective dimension.
Link Prediction in Weighted Networks: A Weighted Mutual Information Model
Zhu, Boyao; Xia, Yongxiang
2016-01-01
The link-prediction problem is an open issue in data mining and knowledge discovery, which attracts researchers from disparate scientific communities. A wealth of methods have been proposed to deal with this problem. Among these approaches, most are applied in unweighted networks, with only a few taking the weights of links into consideration. In this paper, we present a weighted model for undirected and weighted networks based on the mutual information of local network structures, where link weights are applied to further enhance the distinguishable extent of candidate links. Empirical experiments are conducted on four weighted networks, and results show that the proposed method can provide more accurate predictions than not only traditional unweighted indices but also typical weighted indices. Furthermore, some in-depth discussions on the effects of weak ties in link prediction as well as the potential to predict link weights are also given. This work may shed light on the design of algorithms for link prediction in weighted networks. PMID:26849659
Link Prediction in Weighted Networks: A Weighted Mutual Information Model.
Zhu, Boyao; Xia, Yongxiang
2016-01-01
The link-prediction problem is an open issue in data mining and knowledge discovery, which attracts researchers from disparate scientific communities. A wealth of methods have been proposed to deal with this problem. Among these approaches, most are applied in unweighted networks, with only a few taking the weights of links into consideration. In this paper, we present a weighted model for undirected and weighted networks based on the mutual information of local network structures, where link weights are applied to further enhance the distinguishable extent of candidate links. Empirical experiments are conducted on four weighted networks, and results show that the proposed method can provide more accurate predictions than not only traditional unweighted indices but also typical weighted indices. Furthermore, some in-depth discussions on the effects of weak ties in link prediction as well as the potential to predict link weights are also given. This work may shed light on the design of algorithms for link prediction in weighted networks. PMID:26849659
Mutual Information for the Detection of Crush
Harding, Peter; Gwynne, Steve; Amos, Martyn
2011-01-01
Fatal crush conditions occur in crowds with tragic frequency. Event organizers and architects are often criticised for failing to consider the causes and implications of crush, but the reality is that both the prediction and prevention of such conditions offer a significant technical challenge. Full treatment of physical force within crowd simulations is precise but often computationally expensive; the more common method of human interpretation of results is computationally “cheap” but subjective and time-consuming. This paper describes an alternative method for the analysis of crowd behaviour, which uses information theory to measure crowd disorder. We show how this technique may be easily incorporated into an existing simulation framework, and validate it against an historical event. Our results show that this method offers an effective and efficient route towards automatic detection of the onset of crush. PMID:22229055
Mutual information area laws for thermal free fermions
NASA Astrophysics Data System (ADS)
Bernigau, H.; Kastoryano, M. J.; Eisert, J.
2015-02-01
We provide a rigorous and asymptotically exact expression of the mutual information of translationally invariant free fermionic lattice systems in a Gibbs state. In order to arrive at this result, we introduce a novel framework for computing determinants of Töplitz operators with smooth symbols, and for treating Töplitz matrices with system size dependent entries. The asymptotically exact mutual information for a partition of the 1D lattice satisfies an area law, with a prefactor which we compute explicitly. As examples, we discuss the fermionic XX model in one dimension and free fermionic models on the torus in higher dimensions in detail. Special emphasis is put on the discussion of the temperature dependence of the mutual information, scaling like the logarithm of the inverse temperature, hence confirming an expression suggested by conformal field theory. We also comment on the applicability of the formalism to treat open systems driven by quantum noise. In the appendix, we derive useful bounds to the mutual information in terms of purities. Finally, we provide a detailed error analysis for finite system sizes. This analysis is valuable in its own right for the abstract theory of Töplitz determinants.
Using Mutual Information for Adaptive Item Comparison and Student Assessment
ERIC Educational Resources Information Center
Liu, Chao-Lin
2005-01-01
The author analyzes properties of mutual information between dichotomous concepts and test items. The properties generalize some common intuitions about item comparison, and provide principled foundations for designing item-selection heuristics for student assessment in computer-assisted educational systems. The proposed item-selection strategies…
Mutual Information Item Selection in Adaptive Classification Testing
ERIC Educational Resources Information Center
Weissman, Alexander
2007-01-01
A general approach for item selection in adaptive multiple-category classification tests is provided. The approach uses mutual information (MI), a special case of the Kullback-Leibler distance, or relative entropy. MI works efficiently with the sequential probability ratio test and alleviates the difficulties encountered with using other local-…
Two-Level Document Ranking Using Mutual Information in Natural Language Information Retrieval.
ERIC Educational Resources Information Center
Kang, Hyun-Kyu; Choi, Key-Sun
1997-01-01
Discussion of information retrieval and relevance focuses on mutual information, a measure which represents the relation between two words. A model of a natural-language information-retrieval system that is based on a two-level document-ranking method using mutual information is presented, and a Korean encyclopedia test collection is explained.…
On the Time Complexity of Dijkstra's Three-State Mutual Exclusion Algorithm
NASA Astrophysics Data System (ADS)
Kimoto, Masahiro; Tsuchiya, Tatsuhiro; Kikuno, Tohru
In this letter we give a lower bound on the worst-case time complexity of Dijkstra's three-state mutual exclusion algorithm by specifying a concrete behavior of the algorithm. We also show that our result is more accurate than the known best bound.
Extracting an entanglement signature from only classical mutual information
Starling, David J.; Howell, John C.; Broadbent, Curtis J.
2011-09-15
We introduce a quantity which is formed using classical notions of mutual information and which is computed using the results of projective measurements. This quantity constitutes a sufficient condition for entanglement and represents the amount of information that can be extracted from a bipartite system for spacelike separated observers. In addition to discussion, we provide simulations as well as experimental results for the singlet and maximally correlated mixed states.
Fatakia, Sarosh N; Costanzi, Stefano; Chow, Carson C
2009-01-01
G protein-coupled receptors (GPCRs) are a superfamily of seven transmembrane-spanning proteins involved in a wide array of physiological functions and are the most common targets of pharmaceuticals. This study aims to identify a cohort or clique of positions that share high mutual information. Using a multiple sequence alignment of the transmembrane (TM) domains, we calculated the mutual information between all inter-TM pairs of aligned positions and ranked the pairs by mutual information. A mutual information graph was constructed with vertices that corresponded to TM positions and edges between vertices were drawn if the mutual information exceeded a threshold of statistical significance. Positions with high degree (i.e. had significant mutual information with a large number of other positions) were found to line a well defined inter-TM ligand binding cavity for class A as well as class C GPCRs. Although the natural ligands of class C receptors bind to their extracellular N-terminal domains, the possibility of modulating their activity through ligands that bind to their helical bundle has been reported. Such positions were not found for class B GPCRs, in agreement with the observation that there are not known ligands that bind within their TM helical bundle. All identified key positions formed a clique within the MI graph of interest. For a subset of class A receptors we also considered the alignment of a portion of the second extracellular loop, and found that the two positions adjacent to the conserved Cys that bridges the loop with the TM3 qualified as key positions. Our algorithm may be useful for localizing topologically conserved regions in other protein families. PMID:19262747
Motion Estimation Based on Mutual Information and Adaptive Multi-Scale Thresholding.
Xu, Rui; Taubman, David; Naman, Aous Thabit
2016-03-01
This paper proposes a new method of calculating a matching metric for motion estimation. The proposed method splits the information in the source images into multiple scale and orientation subbands, reduces the subband values to a binary representation via an adaptive thresholding algorithm, and uses mutual information to model the similarity of corresponding square windows in each image. A moving window strategy is applied to recover a dense estimated motion field whose properties are explored. The proposed matching metric is a sum of mutual information scores across space, scale, and orientation. This facilitates the exploitation of information diversity in the source images. Experimental comparisons are performed amongst several related approaches, revealing that the proposed matching metric is better able to exploit information diversity, generating more accurate motion fields. PMID:26742132
NASA Astrophysics Data System (ADS)
Grieggs, Samuel M.; McLaughlin, Michael J.; Ezekiel, Soundararajan; Blasch, Erik
2015-06-01
As technology and internet use grows at an exponential rate, video and imagery data is becoming increasingly important. Various techniques such as Wide Area Motion imagery (WAMI), Full Motion Video (FMV), and Hyperspectral Imaging (HSI) are used to collect motion data and extract relevant information. Detecting and identifying a particular object in imagery data is an important step in understanding visual imagery, such as content-based image retrieval (CBIR). Imagery data is segmented and automatically analyzed and stored in dynamic and robust database. In our system, we seek utilize image fusion methods which require quality metrics. Many Image Fusion (IF) algorithms have been proposed based on different, but only a few metrics, used to evaluate the performance of these algorithms. In this paper, we seek a robust, objective metric to evaluate the performance of IF algorithms which compares the outcome of a given algorithm to ground truth and reports several types of errors. Given the ground truth of a motion imagery data, it will compute detection failure, false alarm, precision and recall metrics, background and foreground regions statistics, as well as split and merge of foreground regions. Using the Structural Similarity Index (SSIM), Mutual Information (MI), and entropy metrics; experimental results demonstrate the effectiveness of the proposed methodology for object detection, activity exploitation, and CBIR.
Scale-Space Mutual Information for Textural-Patterns Characterization
Seedahmed, Gamal H.; Ward, Andy L.
2005-08-22
The essence of image texture is typically understood by two aspects. First, within a texture-pattern there is a significant variation in intensity values between nearby pixels. Second, texture is a homogeneous property at some spatial scale larger than the spatial resolution of the image. Motivated by the essential aspects of image texture, this paper proposes a novel methodology that combines the use of scale-space and mutual information to characterize textural-patterns. Scale-space offers the mechanism for a multi-scale representation of the image, which will be used to address the scale aspect of texture. On the other hand, mutual information provides a measure to quantify the dependency relationship across the scale-space. It has been found that the proposed methodology has the potential to capture different properties of texture such as periodicity, scale, fineness, coarseness, and spatial extent or size. Practical examples are provided to demonstrate the applicability of the proposed methodology.
Entanglement entropy and mutual information in Bose-Einstein condensates
Ding Wenxin; Yang Kun
2009-07-15
In this paper we study the entanglement properties of free nonrelativistic Bose gases. At zero temperature, we calculate the bipartite block entanglement entropy of the system and find that it diverges logarithmically with the particle number in the subsystem. For finite temperatures, we study the mutual information between the two blocks. We first analytically study an infinite-range hopping model, then numerically study a set of long-range hopping models in one dimension that exhibit Bose-Einstein condensation. In both cases we find that a Bose-Einstein condensate, if present, makes a divergent contribution to the mutual information which is proportional to the logarithm of the number of particles in the condensate in the subsystem. The prefactor of the logarithmic divergent term is model dependent.
Mutual information as an order parameter for quantum synchronization
NASA Astrophysics Data System (ADS)
Ameri, V.; Eghbali-Arani, M.; Mari, A.; Farace, A.; Kheirandish, F.; Giovannetti, V.; Fazio, R.
2015-01-01
Spontaneous synchronization is a fundamental phenomenon, important in many theoretical studies and applications. Recently, this effect has been analyzed and observed in a number of physical systems close to the quantum-mechanical regime. In this work we propose mutual information as a useful order parameter which can capture the emergence of synchronization in very different contexts, ranging from semiclassical to intrinsically quantum-mechanical systems. Specifically, we first study the synchronization of two coupled Van der Pol oscillators in both classical and quantum regimes and later we consider the synchronization of two qubits inside two coupled optical cavities. In all these contexts, we find that mutual information can be used as an appropriate figure of merit for determining the synchronization phases independently of the specific details of the system.
Networks in financial markets based on the mutual information rate
NASA Astrophysics Data System (ADS)
Fiedor, Paweł
2014-05-01
In the last few years there have been many efforts in econophysics studying how network theory can facilitate understanding of complex financial markets. These efforts consist mainly of the study of correlation-based hierarchical networks. This is somewhat surprising as the underlying assumptions of research looking at financial markets are that they are complex systems and thus behave in a nonlinear manner, which is confirmed by numerous studies, making the use of correlations which are inherently dealing with linear dependencies only baffling. In this paper we introduce a way to incorporate nonlinear dynamics and dependencies into hierarchical networks to study financial markets using mutual information and its dynamical extension: the mutual information rate. We show that this approach leads to different results than the correlation-based approach used in most studies, on the basis of 91 companies listed on the New York Stock Exchange 100 between 2003 and 2013, using minimal spanning trees and planar maximally filtered graphs.
Part mutual information for quantifying direct associations in networks.
Zhao, Juan; Zhou, Yiwei; Zhang, Xiujun; Chen, Luonan
2016-05-01
Quantitatively identifying direct dependencies between variables is an important task in data analysis, in particular for reconstructing various types of networks and causal relations in science and engineering. One of the most widely used criteria is partial correlation, but it can only measure linearly direct association and miss nonlinear associations. However, based on conditional independence, conditional mutual information (CMI) is able to quantify nonlinearly direct relationships among variables from the observed data, superior to linear measures, but suffers from a serious problem of underestimation, in particular for those variables with tight associations in a network, which severely limits its applications. In this work, we propose a new concept, "partial independence," with a new measure, "part mutual information" (PMI), which not only can overcome the problem of CMI but also retains the quantification properties of both mutual information (MI) and CMI. Specifically, we first defined PMI to measure nonlinearly direct dependencies between variables and then derived its relations with MI and CMI. Finally, we used a number of simulated data as benchmark examples to numerically demonstrate PMI features and further real gene expression data from Escherichia coli and yeast to reconstruct gene regulatory networks, which all validated the advantages of PMI for accurately quantifying nonlinearly direct associations in networks. PMID:27092000
Track-to-Track Data Association using Mutual Information
NASA Astrophysics Data System (ADS)
Hussein, I.; Roscoe, C.; Wilkins, M.; Schumacher, P.
In this paper, we build on recent work to further investigate the use of mutual information to solve various association problems in space situational awareness. Specifically, we solve the track-to-track association (TTTA) problem where we seek to associate a given set of tracks at one point in time with another set of tracks at a different time instance. Both sets of tracks are uncertain and are probabilistically described using multivariate normal distributions. This allows for a closed-form solution, based on the unscented transform and on mutual information. Future work will focus on developing a similar solution when uncertainty is analytic but not Gaussian or when it is completely non-analytic -e.g., when the uncertainty is described using a particle cloud. The proposed solution is inspired by a similar solution based on the unscented transform and mutual information for the observation-to-observation association (OTOA) problem that was developed by the authors in the past. This solution can be adjusted to address the classical observation-to-track association problem (OTTA), which will be the focus of future research. We will demonstrate the main result in simulation for LEO, MEO, GTO, and GEO orbit regimes to show general applicability.
Anisotropic magnetotelluric inversion using a mutual information constraint
NASA Astrophysics Data System (ADS)
Mandolesi, E.; Jones, A. G.
2012-12-01
In recent years, several authors pointed that the electrical conductivity of many subsurface structures cannot be described properly by a scalar field. With the development of field devices and techniques, data quality improved to the point that the anisotropy in conductivity of rocks (microscopic anisotropy) and tectonic structures (macroscopic anisotropy) cannot be neglected. Therefore a correct use of high quality data has to include electrical anisotropy and a correct interpretation of anisotropic data characterizes directly a non-negligible part of the subsurface. In this work we test an inversion routine that takes advantage of the classic Levenberg-Marquardt (LM) algorithm to invert magnetotelluric (MT) data generated from a bi-dimensional (2D) anisotropic domain. The LM method is routinely used in inverse problems due its performance and robustness. In non-linear inverse problems -such the MT problem- the LM method provides a spectacular compromise betwee quick and secure convergence at the price of the explicit computation and storage of the sensitivity matrix. Regularization in inverse MT problems has been used extensively, due to the necessity to constrain model space and to reduce the ill-posedness of the anisotropic MT problem, which makes MT inversions extremely challenging. In order to reduce non-uniqueness of the MT problem and to reach a model compatible with other different tomographic results from the same target region, we used a mutual information (MI) based constraint. MI is a basic quantity in information theory that can be used to define a metric between images, and it is routinely used in fields as computer vision, image registration and medical tomography, to cite some applications. We -thus- inverted for the model that best fits the anisotropic data and that is the closest -in a MI sense- to a tomographic model of the target area. The advantage of this technique is that the tomographic model of the studied region may be produced by any
Link Prediction in Complex Networks: A Mutual Information Perspective
Tan, Fei; Xia, Yongxiang; Zhu, Boyao
2014-01-01
Topological properties of networks are widely applied to study the link-prediction problem recently. Common Neighbors, for example, is a natural yet efficient framework. Many variants of Common Neighbors have been thus proposed to further boost the discriminative resolution of candidate links. In this paper, we reexamine the role of network topology in predicting missing links from the perspective of information theory, and present a practical approach based on the mutual information of network structures. It not only can improve the prediction accuracy substantially, but also experiences reasonable computing complexity. PMID:25207920
NASA Astrophysics Data System (ADS)
Shi, Jun; Xiao, Zhiheng; Zhou, Shichong
2010-07-01
Image segmentation is very important in the field of image processing. The pulse coupled neural network (PCNN) has been efficiently applied to image processing, especially for image segmentation. In this study, a simplified PCNN (S-PCNN) model is proposed, the fuzzy mutual information (FMI) is improved as optimization criterion for S-PCNN, and then the S-PCNN and improved FMI (IFMI) based segmentation algorithm is proposed and applied for the segmentation of breast tumor in ultrasound image. To validate the proposed algorithm, a comparative experiment is implemented to segment breast images not only by our proposed algorithm, but also by the improved C-V algorithm, the max-entropy-based PCNN algorithm, the MI-based PCNN algorithm, and the IFMI-based PCNN algorithm. The results show that the breast lesions are well segmented by the proposed algorithm without image preprocessing, with the mean Hausdorff of distance of 5.631+/-0.822, mean average minimum Euclidean distance of 0.554+/-0.049, mean Tanimoto coefficient of 0.961+/-0.019, and mean misclassified error of 0.038+/-0.004. These values of evaluation indices are better than those of other segmentation algorithms. The results indicate that the proposed algorithm has excellent segmentation accuracy and strong robustness against noise, and it has the potential for breast ultrasound computer-aided diagnosis (CAD).
Lachmann, Alexander; Giorgi, Federico M.; Lopez, Gonzalo; Califano, Andrea
2016-01-01
Summary: The accurate reconstruction of gene regulatory networks from large scale molecular profile datasets represents one of the grand challenges of Systems Biology. The Algorithm for the Reconstruction of Accurate Cellular Networks (ARACNe) represents one of the most effective tools to accomplish this goal. However, the initial Fixed Bandwidth (FB) implementation is both inefficient and unable to deal with sample sets providing largely uneven coverage of the probability density space. Here, we present a completely new implementation of the algorithm, based on an Adaptive Partitioning strategy (AP) for estimating the Mutual Information. The new AP implementation (ARACNe-AP) achieves a dramatic improvement in computational performance (200× on average) over the previous methodology, while preserving the Mutual Information estimator and the Network inference accuracy of the original algorithm. Given that the previous version of ARACNe is extremely demanding, the new version of the algorithm will allow even researchers with modest computational resources to build complex regulatory networks from hundreds of gene expression profiles. Availability and Implementation: A JAVA cross-platform command line executable of ARACNe, together with all source code and a detailed usage guide are freely available on Sourceforge (http://sourceforge.net/projects/aracne-ap). JAVA version 8 or higher is required. Contact: califano@c2b2.columbia.edu Supplementary information: Supplementary data are available at Bioinformatics online. PMID:27153652
Gaining (mutual) information about quark/gluon discrimination
NASA Astrophysics Data System (ADS)
Larkoski, Andrew J.; Thaler, Jesse; Waalewijn, Wouter J.
2014-11-01
Discriminating quark jets from gluon jets is an important but challenging problem in jet substructure. In this paper, we use the concept of mutual information to illuminate the physics of quark/gluon tagging. Ideal quark/gluon separation requires only one bit of truth information, so even if two discriminant variables are largely uncorrelated, they can still share the same "truth overlap". Mutual information can be used to diagnose such situations, and thus determine which discriminant variables are redundant and which can be combined to improve performance. Using both parton showers and analytic resummation, we study a two-parameter family of generalized angularities, which includes familiar infrared and collinear (IRC) safe observables like thrust and broadening, as well as IRC unsafe variants like p {/T D } and hadron multiplicity. At leading-logarithmic (LL) order, the bulk of these variables exhibit Casimir scaling, such that their truth overlap is a universal function of the color factor ratio C A /C F . Only at next-to-leading-logarithmic (NLL) order can one see a difference in quark/gluon performance. For the IRC safe angularities, we show that the quark/gluon performance can be improved by combining angularities with complementary angular exponents. Interestingly, LL order, NLL order, Pythia 8, and Herwig++ all exhibit similar correlations between observables, but there are significant differences in the predicted quark/gluon discrimination power. For the IRC unsafe angularities, we show that the mutual information can be calculated analytically with the help of a nonperturbative "weighted-energy function", providing evidence for the complementarity of safe and unsafe observables for quark/gluon discrimination.
Permutation auto-mutual information of electroencephalogram in anesthesia
NASA Astrophysics Data System (ADS)
Liang, Zhenhu; Wang, Yinghua; Ouyang, Gaoxiang; Voss, Logan J.; Sleigh, Jamie W.; Li, Xiaoli
2013-04-01
Objective. The dynamic change of brain activity in anesthesia is an interesting topic for clinical doctors and drug designers. To explore the dynamical features of brain activity in anesthesia, a permutation auto-mutual information (PAMI) method is proposed to measure the information coupling of electroencephalogram (EEG) time series obtained in anesthesia. Approach. The PAMI is developed and applied on EEG data collected from 19 patients under sevoflurane anesthesia. The results are compared with the traditional auto-mutual information (AMI), SynchFastSlow (SFS, derived from the BIS index), permutation entropy (PE), composite PE (CPE), response entropy (RE) and state entropy (SE). Performance of all indices is assessed by pharmacokinetic/pharmacodynamic (PK/PD) modeling and prediction probability. Main results. The PK/PD modeling and prediction probability analysis show that the PAMI index correlates closely with the anesthetic effect. The coefficient of determination R2 between PAMI values and the sevoflurane effect site concentrations, and the prediction probability Pk are higher in comparison with other indices. The information coupling in EEG series can be applied to indicate the effect of the anesthetic drug sevoflurane on the brain activity as well as other indices. The PAMI of the EEG signals is suggested as a new index to track drug concentration change. Significance. The PAMI is a useful index for analyzing the EEG dynamics during general anesthesia.
Estimation and classification by sigmoids based on mutual information
NASA Technical Reports Server (NTRS)
Baram, Yoram
1994-01-01
An estimate of the probability density function of a random vector is obtained by maximizing the mutual information between the input and the output of a feedforward network of sigmoidal units with respect to the input weights. Classification problems can be solved by selecting the class associated with the maximal estimated density. Newton's s method, applied to an estimated density, yields a recursive maximum likelihood estimator, consisting of a single internal layer of sigmoids, for a random variable or a random sequence. Applications to the diamond classification and to the prediction of a sun-spot process are demonstrated.
Mutual information measures applied to EEG signals for sleepiness characterization.
Melia, Umberto; Guaita, Marc; Vallverdú, Montserrat; Embid, Cristina; Vilaseca, Isabel; Salamero, Manel; Santamaria, Joan
2015-03-01
Excessive daytime sleepiness (EDS) is one of the main symptoms of several sleep related disorders with a great impact on the patient lives. While many studies have been carried out in order to assess daytime sleepiness, the automatic EDS detection still remains an open problem. In this work, a novel approach to this issue based on non-linear dynamical analysis of EEG signal was proposed. Multichannel EEG signals were recorded during five maintenance of wakefulness (MWT) and multiple sleep latency (MSLT) tests alternated throughout the day from patients suffering from sleep disordered breathing. A group of 20 patients with excessive daytime sleepiness (EDS) was compared with a group of 20 patients without daytime sleepiness (WDS), by analyzing 60-s EEG windows in waking state. Measures obtained from cross-mutual information function (CMIF) and auto-mutual-information function (AMIF) were calculated in the EEG. These functions permitted a quantification of the complexity properties of the EEG signal and the non-linear couplings between different zones of the scalp. Statistical differences between EDS and WDS groups were found in β band during MSLT events (p-value < 0.0001). WDS group presented more complexity than EDS in the occipital zone, while a stronger nonlinear coupling between occipital and frontal zones was detected in EDS patients than in WDS. The AMIF and CMIF measures yielded sensitivity and specificity above 80% and AUC of ROC above 0.85 in classifying EDS and WDS patients. PMID:25638417
Classical mutual information in mean-field spin glass models
NASA Astrophysics Data System (ADS)
Alba, Vincenzo; Inglis, Stephen; Pollet, Lode
2016-03-01
We investigate the classical Rényi entropy Sn and the associated mutual information In in the Sherrington-Kirkpatrick (S-K) model, which is the paradigm model of mean-field spin glasses. Using classical Monte Carlo simulations and analytical tools we investigate the S-K model in the n -sheet booklet. This is achieved by gluing together n independent copies of the model, and it is the main ingredient for constructing the Rényi entanglement-related quantities. We find a glassy phase at low temperatures, whereas at high temperatures the model exhibits paramagnetic behavior, consistent with the regular S-K model. The temperature of the paramagnetic-glassy transition depends nontrivially on the geometry of the booklet. At high temperatures we provide the exact solution of the model by exploiting the replica symmetry. This is the permutation symmetry among the fictitious replicas that are used to perform disorder averages (via the replica trick). In the glassy phase the replica symmetry has to be broken. Using a generalization of the Parisi solution, we provide analytical results for Sn and In and for standard thermodynamic quantities. Both Sn and In exhibit a volume law in the whole phase diagram. We characterize the behavior of the corresponding densities, Sn/N and In/N , in the thermodynamic limit. Interestingly, at the critical point the mutual information does not exhibit any crossing for different system sizes, in contrast with local spin models.
Mortazavi, Atiyeh; Moattar, Mohammad Hossein
2016-01-01
High dimensionality of microarray data sets may lead to low efficiency and overfitting. In this paper, a multiphase cooperative game theoretic feature selection approach is proposed for microarray data classification. In the first phase, due to high dimension of microarray data sets, the features are reduced using one of the two filter-based feature selection methods, namely, mutual information and Fisher ratio. In the second phase, Shapley index is used to evaluate the power of each feature. The main innovation of the proposed approach is to employ Qualitative Mutual Information (QMI) for this purpose. The idea of Qualitative Mutual Information causes the selected features to have more stability and this stability helps to deal with the problem of data imbalance and scarcity. In the third phase, a forward selection scheme is applied which uses a scoring function to weight each feature. The performance of the proposed method is compared with other popular feature selection algorithms such as Fisher ratio, minimum redundancy maximum relevance, and previous works on cooperative game based feature selection. The average classification accuracy on eleven microarray data sets shows that the proposed method improves both average accuracy and average stability compared to other approaches. PMID:27127506
Naghibi, Tofigh; Hoffmann, Sarah; Pfister, Beat
2015-08-01
Feature subset selection, as a special case of the general subset selection problem, has been the topic of a considerable number of studies due to the growing importance of data-mining applications. In the feature subset selection problem there are two main issues that need to be addressed: (i) Finding an appropriate measure function than can be fairly fast and robustly computed for high-dimensional data. (ii) A search strategy to optimize the measure over the subset space in a reasonable amount of time. In this article mutual information between features and class labels is considered to be the measure function. Two series expansions for mutual information are proposed, and it is shown that most heuristic criteria suggested in the literature are truncated approximations of these expansions. It is well-known that searching the whole subset space is an NP-hard problem. Here, instead of the conventional sequential search algorithms, we suggest a parallel search strategy based on semidefinite programming (SDP) that can search through the subset space in polynomial time. By exploiting the similarities between the proposed algorithm and an instance of the maximum-cut problem in graph theory, the approximation ratio of this algorithm is derived and is compared with the approximation ratio of the backward elimination method. The experiments show that it can be misleading to judge the quality of a measure solely based on the classification accuracy, without taking the effect of the non-optimum search strategy into account. PMID:26352993
Long-range mutual information and topological uncertainty principle
NASA Astrophysics Data System (ADS)
Jian, Chao-Ming; Kim, Isaac; Qi, Xiao-Liang
Ordered phases in Landau paradigm can be diagnosed by a local order parameter, whereas topologically ordered phases cannot be detected in such a way. In this paper, we propose long-range mutual information (LRMI) as a unified diagnostic for both conventional long-range order and topological order. Using the LRMI, we characterize orders in n +1D gapped systems as m-membrane condensates with 0 <= m <= n-1. The familiar conventional order and 2 +1D topological orders are respectively identified as 0-membrane and 1-membrane condensates. We propose and study the topological uncertainty principle, which describes the non-commuting nature of non-local order parameters in topological orders.
Extending the scope of holographic mutual information and chaotic behavior
NASA Astrophysics Data System (ADS)
Sircar, Nilanjan; Sonnenschein, Jacob; Tangarife, Walter
2016-05-01
We extend the use of holography to investigate the scrambling properties of various physical systems. Specifically, we consider: (i) non-conformal backgrounds of black Dp branes, (ii) asymptotically Lifshitz black holes, and (iii) black AdS solutions of Gauss-Bonnet gravity. We use the disruption of the entanglement entropy as a probe of the chaotic features of such systems. Our analysis shows that these theories share the same type of behavior as conformal theories as they undergo chaos; however, in the case of Gauss-Bonnet gravity, we find a stark difference in the evolution of the mutual information for negative Gauss-Bonnet coupling. This may signal an inconsistency of the latter.
Mutual information as a measure of image quality for 3D dynamic lung imaging with EIT
Crabb, M G; Davidson, J L; Little, R; Wright, P; Morgan, A R; Miller, C A; Naish, J H; Parker, G J M; Kikinis, R; McCann, H; Lionheart, W R B
2014-01-01
We report on a pilot study of dynamic lung electrical impedance tomography (EIT) at the University of Manchester. Low-noise EIT data at 100 frames per second (fps) were obtained from healthy male subjects during controlled breathing, followed by magnetic resonance imaging (MRI) subsequently used for spatial validation of the EIT reconstruction. The torso surface in the MR image and electrode positions obtained using MRI fiducial markers informed the construction of a 3D finite element model extruded along the caudal-distal axis of the subject. Small changes in the boundary that occur during respiration were accounted for by incorporating the sensitivity with respect to boundary shape into a robust temporal difference reconstruction algorithm. EIT and MRI images were co-registered using the open source medical imaging software, 3D Slicer. A quantitative comparison of quality of different EIT reconstructions was achieved through calculation of the mutual information with a lung-segmented MR image. EIT reconstructions using a linear shape correction algorithm reduced boundary image artefacts, yielding better contrast of the lungs, and had 10% greater mutual information compared with a standard linear EIT reconstruction. PMID:24710978
Optimal and Suboptimal Noises Enhancing Mutual Information in Threshold System
NASA Astrophysics Data System (ADS)
Zhai, Qiqing; Wang, Youguo
2016-05-01
In this paper, we investigate the efficacy of noise enhancing information transmission in a threshold system. At first, in the frame of stochastic resonance (SR), optimal noise (Opt N) is derived to maximize mutual information (MI) of this nonlinear system. When input signal is discrete (binary), the optimal SR noise is found to have a finite distribution. In contrast, when input signal is continuous, the optimal SR noise is a constant one. In addition, suboptimal SR noises are explored as well with optimization methods when the types of noise added into the system are predetermined. We find that for small thresholds, suboptimal noises do not exist. Only when thresholds reach some level, do suboptimal noises come into effect. Meanwhile, we have discussed the impact of tails in noise distribution on SR effect. Finally, this paper extends the single-threshold system to an array of multi-threshold devices and presents the corresponding efficacy of information transmission produced by optimal and suboptimal SR noises. These results may be beneficial to quantization and coding.
Mutual information-based feature selection for radiomics
NASA Astrophysics Data System (ADS)
Oubel, Estanislao; Beaumont, Hubert; Iannessi, Antoine
2016-03-01
Background The extraction and analysis of image features (radiomics) is a promising field in the precision medicine era, with applications to prognosis, prediction, and response to treatment quantification. In this work, we present a mutual information - based method for quantifying reproducibility of features, a necessary step for qualification before their inclusion in big data systems. Materials and Methods Ten patients with Non-Small Cell Lung Cancer (NSCLC) lesions were followed over time (7 time points in average) with Computed Tomography (CT). Five observers segmented lesions by using a semi-automatic method and 27 features describing shape and intensity distribution were extracted. Inter-observer reproducibility was assessed by computing the multi-information (MI) of feature changes over time, and the variability of global extrema. Results The highest MI values were obtained for volume-based features (VBF). The lesion mass (M), surface to volume ratio (SVR) and volume (V) presented statistically significant higher values of MI than the rest of features. Within the same VBF group, SVR showed also the lowest variability of extrema. The correlation coefficient (CC) of feature values was unable to make a difference between features. Conclusions MI allowed to discriminate three features (M, SVR, and V) from the rest in a statistically significant manner. This result is consistent with the order obtained when sorting features by increasing values of extrema variability. MI is a promising alternative for selecting features to be considered as surrogate biomarkers in a precision medicine context.
Automatic Registration of Multi-Source Data Using Mutual Information
NASA Astrophysics Data System (ADS)
Parmehr, E. G.; Zhang, C.; Fraser, C. S.
2012-07-01
Automatic image registration is a basic step in multi-sensor data integration in remote sensing and photogrammetric applications such as data fusion. The effectiveness of Mutual Information (MI) as a technique for automated multi-sensor image registration has previously been demonstrated for medical and remote sensing applications. In this paper, a new General Weighted MI (GWMI) approach that improves the robustness of MI to local maxima, particularly in the case of registering optical imagery and 3D point clouds, is presented. Two different methods including a Gaussian Mixture Model (GMM) and Kernel Density Estimation have been used to define the weight function of joint probability, regardless of the modality of the data being registered. The Expectation Maximizing method is then used to estimate parameters of GMM, and in order to reduce the cost of computation, a multi-resolution strategy has been used. The performance of the proposed GWMI method for the registration of aerial orthotoimagery and LiDAR range and intensity information has been experimentally evaluated and the results obtained are presented.
Mutual Algorithm-Architecture Analysis for Real - Parallel Systems in Particle Physics Experiments.
NASA Astrophysics Data System (ADS)
Ni, Ping
Data acquisition from particle colliders requires real-time detection of tracks and energy clusters from collision events occurring at intervals of tens of mus. Beginning with the specification of a benchmark track-finding algorithm, parallel implementations have been developed. A revision of the routing scheme for performing reductions such as a tree sum, called the reduced routing distance scheme, has been developed and analyzed. The scheme reduces inter-PE communication time for narrow communication channel systems. A new parallel algorithm, called the interleaved tree sum, for parallel reduction problems has been developed that increases efficiency of processor use. Detailed analysis of this algorithm with different routing schemes is presented. Comparable parallel algorithms are analyzed, also taking into account the architectural parameters that play an important role in this parallel algorithm analysis. Computation and communication times are analyzed to guide the design of a custom system based on a massively parallel processing component. Developing an optimal system requires mutual analysis of algorithm and architecture parameters. It is shown that matching a processor array size to the parallelism of the problem does not always produce the best system design. Based on promising benchmark simulation results, an application specific hardware prototype board, called Dasher, has been built using two Blitzen chips. The processing array is a mesh-connected SIMD system with 256 PEs. Its design is discussed, with details on the software environment.
A mutual-information-based mining method for marine abnormal association rules
NASA Astrophysics Data System (ADS)
Cunjin, Xue; Wanjiao, Song; Lijuan, Qin; Qing, Dong; Xiaoyang, Wen
2015-03-01
Long time series of remote sensing images are a key source of data for exploring large-scale marine abnormal association patterns, but pose significant challenges for traditional approaches to spatiotemporal analysis. This paper proposes a mutual-information-based quantitative association rule-mining algorithm (MIQarma) to address these challenges. MIQarma comprises three key steps. First, MIQarma calculates the asymmetrical mutual information between items with one scan of the database, and extracts pair-wise related items according to the user-specified information threshold. Second, a linking-pruning-generating recursive loop generates (k+1)-dimensional candidate association rules from k-dimensional rules on basis of the user-specified minimum support threshold, and this step is repeated until no more candidate association rules are generated. Finally, strong association rules are generated according to the user-specified minimum evaluation indicators. To demonstrate the feasibility and efficiency of MIQarma, we present two case studies: one considers performance analysis and the other identifies marine abnormal association relationships.
Solar flux forecasting using mutual information with an optimal delay
NASA Technical Reports Server (NTRS)
Ashrafi, S.; Conway, D.; Rokni, M.; Sperling, R.; Roszman, L.; Cooley, J.
1993-01-01
Solar flux F(sub 10.7) directly affects the atmospheric density, thereby changing the lifetime and prediction of satellite orbits. For this reason, accurate forecasting of F(sub 10.7) is crucial for orbit determination of spacecraft. Our attempts to model and forecast F(sub 10.7) uncovered highly entangled dynamics. We concluded that the general lack of predictability in solar activity arises from its nonlinear nature. Nonlinear dynamics allow us to predict F(sub 10.7) more accurately than is possible using stochastic methods for time scales shorter than a characteristic horizon, and with about the same accuracy as using stochastic techniques when the forecasted data exceed this horizon. The forecast horizon is a function of two dynamical invariants: the attractor dimension and the Lyapunov exponent. In recent years, estimation of the attractor dimension reconstructed from a time series has become an important tool in data analysis. In calculating the invariants of the system, the first necessary step is the reconstruction of the attractor for the system from the time-delayed values of the time series. The choice of the time delay is critical for this reconstruction. For an infinite amount of noise-free data, the time delay can, in principle, be chosen almost arbitrarily. However, the quality of the phase portraits produced using the time-delay technique is determined by the value chosen for the delay time. Fraser and Swinney have shown that a good choice for this time delay is the one suggested by Shaw, which uses the first local minimum of the mutual information rather than the autocorrelation function to determine the time delay. This paper presents a refinement of this criterion and applies the refined technique to solar flux data to produce a forecast of the solar activity.
CSMMI: class-specific maximization of mutual information for action and gesture recognition.
Wan, Jun; Athitsos, Vassilis; Jangyodsuk, Pat; Escalante, Hugo Jair; Ruan, Qiuqi; Guyon, Isabelle
2014-07-01
In this paper, we propose a novel approach called class-specific maximization of mutual information (CSMMI) using a submodular method, which aims at learning a compact and discriminative dictionary for each class. Unlike traditional dictionary-based algorithms, which typically learn a shared dictionary for all of the classes, we unify the intraclass and interclass mutual information (MI) into an single objective function to optimize class-specific dictionary. The objective function has two aims: 1) maximizing the MI between dictionary items within a specific class (intrinsic structure) and 2) minimizing the MI between the dictionary items in a given class and those of the other classes (extrinsic structure). We significantly reduce the computational complexity of CSMMI by introducing an novel submodular method, which is one of the important contributions of this paper. This paper also contributes a state-of-the-art end-to-end system for action and gesture recognition incorporating CSMMI, with feature extraction, learning initial dictionary per each class by sparse coding, CSMMI via submodularity, and classification based on reconstruction errors. We performed extensive experiments on synthetic data and eight benchmark data sets. Our experimental results show that CSMMI outperforms shared dictionary methods and that our end-to-end system is competitive with other state-of-the-art approaches. PMID:24983106
NASA Astrophysics Data System (ADS)
Litva, John; Zeytinoglu, Mehmet
1990-07-01
A study of the effects of mutual coupling on the performance on direction finding algorithms is presented. The MUltiple SIgnal Classification (MUSIC) and maximum likelihood (ML) algorithms resolved non-coherent and coherent incident wave fields with relative ease when ideal array response models were used. When array output vectors were simulated using free space and lossy ground models which include mutual coupling effects, the performance of the unmodified direction finding algorithms declined sharply. In particular, estimates from unmodified algorithms exhibited a constant, deterministic bias term. Algorithms modified for this bias term displayed a significant residual bias in attempts to resolve incident wave field with closely spaced source signals. Array steering vectors are the main components through which the array models are coupled. Free space and lossy ground response modes that take mutual coupling effects into consideration were modified by assembling array steering vectors directly from simulated raw data. These modified algorithms exhibited exemplary performance in resolving fields from closely spaced source signals when the array output vectors were derived from the same free space or lossy ground models. The MUSIC algorithm permitted array steering vectors modified with free space response data to be used with the array output derived for the lossy ground response mode. However, it was applicable only to non-coherent source signals. The ML algorithm performed equally well with non-correlated and coherent source signals but it had to be perfectly matched with the actual array response. Computation requirements were very demanding for the ML algorithm.
Worst-case mutual information trajectories in concatenated codes with asymptotic interleavers
NASA Technical Reports Server (NTRS)
Divsalar, D.; Shamai, S.
2002-01-01
In this summary we present extremal average mutual information values carried by the extrinsic loglikelihood under the constraint of a given mean and variance while accounting for the consistency feature of loglikelihoods.
NASA Astrophysics Data System (ADS)
Zheng, Fawen; Wan, Yongshan; Song, Keunyea; Sun, Detong; Hedgepeth, Marion
2016-01-01
Soil pore water salinity plays an important role in the distribution of vegetation and biogeochemical processes in coastal floodplain ecosystems. In this study, artificial neural networks (ANNs) were applied to simulate the pore water salinity of a tidal floodplain in Florida. We present an approach based on embedding theory with mutual information to reconstruct ANN model input time series from one system state variable. Mutual information between system output and input was computed and the local minimum mutual information points were used to determine a time lag vector for time series embedding and reconstruction, with which the mutual information weighted average method was developed to compute the components of reconstructed time series. The optimal embedding dimension was obtained by optimizing model performance. The method was applied to simulate soil pore water salinity dynamics at 12 probe locations in the tidal floodplain influenced by saltwater intrusion using 4 years (2005-2008) data, in which adjacent river water salinity was used to reconstruct model input. The simulated electrical conductivity of the pore water showed close agreement with field observations (RMSE and ), suggesting the reconstructed input by the proposed approach provided adequate input information for ANN modeling. Multiple linear regression model, partial mutual information algorithm for input variable selection, k-NN algorithm, and simple time delay embedding were also used to further verify the merit of the proposed approach.
NASA Astrophysics Data System (ADS)
Zhang, Yongfei; Cao, Haiheng; Jiang, Hongxu; Li, Bo
2016-04-01
As remote sensing image applications are often characterized with limited bandwidth and high-quality demands, higher coding performance of remote sensing images are desirable. The embedded block coding with optimal truncation (EBCOT) is the fundamental part of JPEG2000 image compression standard. However, EBCOT only considers correlation within a sub-band and utilizes a context template of eight spatially neighboring coefficients in prediction. The existing optimization methods in literature using the current context template prove little performance improvements. To address this problem, this paper presents a new mutual information (MI)-based context template selection and modeling method. By further considering the correlation across the sub-bands, the potential prediction coefficients, including neighbors, far neighbors, parent and parent neighbors, are comprehensively examined and selected in such a manner that achieves a nice trade-off between the MI-based correlation criterion and the prediction complexity. Based on the selected context template, a high-order prediction model, which jointly considers the weight and the significance state of each coefficient, is proposed. Experimental results show that the proposed algorithm consistently outperforms the benchmark JPEG2000 standard and state-of-the-art algorithms in term of coding efficiency at a competitive computational cost, which makes it desirable in real-time compression applications, especially for remote sensing images.
Temporal subtraction in chest radiography: Mutual information as a measure of image quality
Armato, Samuel G. III; Sensakovic, William F.; Passen, Samantha J.; Engelmann, Roger; MacMahon, Heber
2009-12-15
Purpose: Temporal subtraction is used to detect the interval change in chest radiographs and aid radiologists in patient diagnosis. This method registers two temporally different images by geometrically warping the lung region, or ''lung mask,'' of a previous radiographic image to align with the current image. The gray levels of every pixel in the current image are subtracted from the gray levels of the corresponding pixels in the warped previous image to form a temporal subtraction image. While temporal subtraction images effectively enhance areas of pathologic change, misregistration of the images can mislead radiologists by obscuring the interval change or by creating artifacts that mimic change. The purpose of this study was to investigate the utility of mutual information computed between two registered radiographic chest images as a metric for distinguishing between clinically acceptable and clinically unacceptable temporal subtraction images.Methods: A radiologist subjectively rated the image quality of 138 temporal subtraction images using a 1 (poor) to 5 (excellent) scale. To objectively assess the registration accuracy depicted in the temporal subtraction images, which is the main factor that affects the quality of these images, mutual information was computed on the two constituent registered images prior to their subtraction to generate a temporal subtraction image. Mutual information measures the joint entropy of the current image and the warped previous image, yielding a higher value when the gray levels of spatially matched pixels in each image are consistent. Mutual information values were correlated with the radiologist's subjective ratings. To improve this correlation, mutual information was computed from a spatially limited lung mask, which was cropped from the bottom by 10%-60%. Additionally, the number of gray-level values used in the joint entropy histogram was varied. The ability of mutual information to predict the clinical acceptability of
Mutual information and the fidelity of response of gene regulatory models
NASA Astrophysics Data System (ADS)
Tabbaa, Omar P.; Jayaprakash, C.
2014-08-01
We investigate cellular response to extracellular signals by using information theory techniques motivated by recent experiments. We present results for the steady state of the following gene regulatory models found in both prokaryotic and eukaryotic cells: a linear transcription-translation model and a positive or negative auto-regulatory model. We calculate both the information capacity and the mutual information exactly for simple models and approximately for the full model. We find that (1) small changes in mutual information can lead to potentially important changes in cellular response and (2) there are diminishing returns in the fidelity of response as the mutual information increases. We calculate the information capacity using Gillespie simulations of a model for the TNF-α-NF-κ B network and find good agreement with the measured value for an experimental realization of this network. Our results provide a quantitative understanding of the differences in cellular response when comparing experimentally measured mutual information values of different gene regulatory models. Our calculations demonstrate that Gillespie simulations can be used to compute the mutual information of more complex gene regulatory models, providing a potentially useful tool in synthetic biology.
NASA Astrophysics Data System (ADS)
Quilty, John; Adamowski, Jan; Khalil, Bahaa; Rathinasamy, Maheswaran
2016-03-01
The input variable selection problem has recently garnered much interest in the time series modeling community, especially within water resources applications, demonstrating that information theoretic (nonlinear)-based input variable selection algorithms such as partial mutual information (PMI) selection (PMIS) provide an improved representation of the modeled process when compared to linear alternatives such as partial correlation input selection (PCIS). PMIS is a popular algorithm for water resources modeling problems considering nonlinear input variable selection; however, this method requires the specification of two nonlinear regression models, each with parametric settings that greatly influence the selected input variables. Other attempts to develop input variable selection methods using conditional mutual information (CMI) (an analog to PMI) have been formulated under different parametric pretenses such as k nearest-neighbor (KNN) statistics or kernel density estimates (KDE). In this paper, we introduce a new input variable selection method based on CMI that uses a nonparametric multivariate continuous probability estimator based on Edgeworth approximations (EA). We improve the EA method by considering the uncertainty in the input variable selection procedure by introducing a bootstrap resampling procedure that uses rank statistics to order the selected input sets; we name our proposed method bootstrap rank-ordered CMI (broCMI). We demonstrate the superior performance of broCMI when compared to CMI-based alternatives (EA, KDE, and KNN), PMIS, and PCIS input variable selection algorithms on a set of seven synthetic test problems and a real-world urban water demand (UWD) forecasting experiment in Ottawa, Canada.
Marrelec, Guillaume; Messé, Arnaud; Bellec, Pierre
2015-01-01
The use of mutual information as a similarity measure in agglomerative hierarchical clustering (AHC) raises an important issue: some correction needs to be applied for the dimensionality of variables. In this work, we formulate the decision of merging dependent multivariate normal variables in an AHC procedure as a Bayesian model comparison. We found that the Bayesian formulation naturally shrinks the empirical covariance matrix towards a matrix set a priori (e.g., the identity), provides an automated stopping rule, and corrects for dimensionality using a term that scales up the measure as a function of the dimensionality of the variables. Also, the resulting log Bayes factor is asymptotically proportional to the plug-in estimate of mutual information, with an additive correction for dimensionality in agreement with the Bayesian information criterion. We investigated the behavior of these Bayesian alternatives (in exact and asymptotic forms) to mutual information on simulated and real data. An encouraging result was first derived on simulations: the hierarchical clustering based on the log Bayes factor outperformed off-the-shelf clustering techniques as well as raw and normalized mutual information in terms of classification accuracy. On a toy example, we found that the Bayesian approaches led to results that were similar to those of mutual information clustering techniques, with the advantage of an automated thresholding. On real functional magnetic resonance imaging (fMRI) datasets measuring brain activity, it identified clusters consistent with the established outcome of standard procedures. On this application, normalized mutual information had a highly atypical behavior, in the sense that it systematically favored very large clusters. These initial experiments suggest that the proposed Bayesian alternatives to mutual information are a useful new tool for hierarchical clustering. PMID:26406245
Marrelec, Guillaume; Messé, Arnaud; Bellec, Pierre
2015-01-01
The use of mutual information as a similarity measure in agglomerative hierarchical clustering (AHC) raises an important issue: some correction needs to be applied for the dimensionality of variables. In this work, we formulate the decision of merging dependent multivariate normal variables in an AHC procedure as a Bayesian model comparison. We found that the Bayesian formulation naturally shrinks the empirical covariance matrix towards a matrix set a priori (e.g., the identity), provides an automated stopping rule, and corrects for dimensionality using a term that scales up the measure as a function of the dimensionality of the variables. Also, the resulting log Bayes factor is asymptotically proportional to the plug-in estimate of mutual information, with an additive correction for dimensionality in agreement with the Bayesian information criterion. We investigated the behavior of these Bayesian alternatives (in exact and asymptotic forms) to mutual information on simulated and real data. An encouraging result was first derived on simulations: the hierarchical clustering based on the log Bayes factor outperformed off-the-shelf clustering techniques as well as raw and normalized mutual information in terms of classification accuracy. On a toy example, we found that the Bayesian approaches led to results that were similar to those of mutual information clustering techniques, with the advantage of an automated thresholding. On real functional magnetic resonance imaging (fMRI) datasets measuring brain activity, it identified clusters consistent with the established outcome of standard procedures. On this application, normalized mutual information had a highly atypical behavior, in the sense that it systematically favored very large clusters. These initial experiments suggest that the proposed Bayesian alternatives to mutual information are a useful new tool for hierarchical clustering. PMID:26406245
Entanglement entropy and mutual information production rates in acoustic black holes.
Giovanazzi, Stefano
2011-01-01
A method to investigate acoustic Hawking radiation is proposed, where entanglement entropy and mutual information are measured from the fluctuations of the number of particles. The rate of entropy radiated per one-dimensional (1D) channel is given by S=κ/12, where κ is the sound acceleration on the sonic horizon. This entropy production is accompanied by a corresponding formation of mutual information to ensure the overall conservation of information. The predictions are confirmed using an ab initio analytical approach in transonic flows of 1D degenerate ideal Fermi fluids. PMID:21231730
Entanglement Entropy and Mutual Information Production Rates in Acoustic Black Holes
Giovanazzi, Stefano
2011-01-07
A method to investigate acoustic Hawking radiation is proposed, where entanglement entropy and mutual information are measured from the fluctuations of the number of particles. The rate of entropy radiated per one-dimensional (1D) channel is given by S={kappa}/12, where {kappa} is the sound acceleration on the sonic horizon. This entropy production is accompanied by a corresponding formation of mutual information to ensure the overall conservation of information. The predictions are confirmed using an ab initio analytical approach in transonic flows of 1D degenerate ideal Fermi fluids.
Monogamy and backflow of mutual information in non-Markovian thermal baths
NASA Astrophysics Data System (ADS)
Costa, A. C. S.; Angelo, R. M.; Beims, M. W.
2014-07-01
We investigate the dynamics of information among the parties of tripartite systems. We start by proving two results concerning the monogamy of mutual information. The first one states that mutual information is monogamous for generic tripartite pure states. The second shows that, in general, mutual information is monogamous only if the amount of genuine tripartite correlations is large enough. Then, we analyze the internal dynamics of tripartite systems whose parties do not exchange energy. In particular, we allow for one of the subsystems to play the role of a finite thermal bath. As a result, we find a typical scenario in which local information tends to be converted into delocalized information. Moreover, we show that (i) the information flow is reversible for finite thermal baths at low temperatures, (ii) monogamy of mutual information is respected throughout the dynamics, and (iii) genuine tripartite correlations are typically present. Finally, we analytically calculate a quantity capable of revealing favorable regimes for non-Markovianity in our model.
What are the differences between Bayesian classifiers and mutual-information classifiers?
Hu, Bao-Gang
2014-02-01
In this paper, both Bayesian and mutual-information classifiers are examined for binary classifications with or without a reject option. The general decision rules are derived for Bayesian classifiers with distinctions on error types and reject types. A formal analysis is conducted to reveal the parameter redundancy of cost terms when abstaining classifications are enforced. The redundancy implies an intrinsic problem of nonconsistency for interpreting cost terms. If no data are given to the cost terms, we demonstrate the weakness of Bayesian classifiers in class-imbalanced classifications. On the contrary, mutual-information classifiers are able to provide an objective solution from the given data, which shows a reasonable balance among error types and reject types. Numerical examples of using two types of classifiers are given for confirming the differences, including the extremely class-imbalanced cases. Finally, we briefly summarize the Bayesian and mutual-information classifiers in terms of their application advantages and disadvantages, respectively. PMID:24807026
The Impact of Different Sources of Fluctuations on Mutual Information in Biochemical Networks
Chevalier, Michael; Venturelli, Ophelia; El-Samad, Hana
2015-01-01
Stochastic fluctuations in signaling and gene expression limit the ability of cells to sense the state of their environment, transfer this information along cellular pathways, and respond to it with high precision. Mutual information is now often used to quantify the fidelity with which information is transmitted along a cellular pathway. Mutual information calculations from experimental data have mostly generated low values, suggesting that cells might have relatively low signal transmission fidelity. In this work, we demonstrate that mutual information calculations might be artificially lowered by cell-to-cell variability in both initial conditions and slowly fluctuating global factors across the population. We carry out our analysis computationally using a simple signaling pathway and demonstrate that in the presence of slow global fluctuations, every cell might have its own high information transmission capacity but that population averaging underestimates this value. We also construct a simple synthetic transcriptional network and demonstrate using experimental measurements coupled to computational modeling that its operation is dominated by slow global variability, and hence that its mutual information is underestimated by a population averaged calculation. PMID:26484538
The Impact of Different Sources of Fluctuations on Mutual Information in Biochemical Networks.
Chevalier, Michael; Venturelli, Ophelia; El-Samad, Hana
2015-10-01
Stochastic fluctuations in signaling and gene expression limit the ability of cells to sense the state of their environment, transfer this information along cellular pathways, and respond to it with high precision. Mutual information is now often used to quantify the fidelity with which information is transmitted along a cellular pathway. Mutual information calculations from experimental data have mostly generated low values, suggesting that cells might have relatively low signal transmission fidelity. In this work, we demonstrate that mutual information calculations might be artificially lowered by cell-to-cell variability in both initial conditions and slowly fluctuating global factors across the population. We carry out our analysis computationally using a simple signaling pathway and demonstrate that in the presence of slow global fluctuations, every cell might have its own high information transmission capacity but that population averaging underestimates this value. We also construct a simple synthetic transcriptional network and demonstrate using experimental measurements coupled to computational modeling that its operation is dominated by slow global variability, and hence that its mutual information is underestimated by a population averaged calculation. PMID:26484538
NASA Astrophysics Data System (ADS)
Tan, Chao; Wang, Jinyue; Wu, Tong; Qin, Xin; Li, Menglong
2010-12-01
Based on the combination of uninformative variable elimination (UVE), bootstrap and mutual information (MI), a simple ensemble algorithm, named ESPLS, is proposed for spectral multivariate calibration (MVC). In ESPLS, those uninformative variables are first removed; and then a preparatory training set is produced by bootstrap, on which a MI spectrum of retained variables is calculated. The variables that exhibit higher MI than a defined threshold form a subspace on which a candidate partial least-squares (PLS) model is constructed. This process is repeated. After a number of candidate models are obtained, a small part of models is picked out to construct an ensemble model by simple/weighted average. Four near/mid-infrared (NIR/MIR) spectral datasets concerning the determination of six components are used to verify the proposed ESPLS. The results indicate that ESPLS is superior to UVEPLS and its combination with MI-based variable selection (SPLS) in terms of both the accuracy and robustness. Besides, from the perspective of end-users, ESPLS does not increase the complexity of a calibration when enhancing its performance.
Identifying relevant group of miRNAs in cancer using fuzzy mutual information.
Pal, Jayanta Kumar; Ray, Shubhra Sankar; Pal, Sankar K
2016-04-01
MicroRNAs (miRNAs) act as a major biomarker of cancer. All miRNAs in human body are not equally important for cancer identification. We propose a methodology, called FMIMS, which automatically selects the most relevant miRNAs for a particular type of cancer. In FMIMS, miRNAs are initially grouped by using a SVM-based algorithm; then the group with highest relevance is determined and the miRNAs in that group are finally ranked for selection according to their redundancy. Fuzzy mutual information is used in computing the relevance of a group and the redundancy of miRNAs within it. Superiority of the most relevant group to all others, in deciding normal or cancer, is demonstrated on breast, renal, colorectal, lung, melanoma and prostate data. The merit of FMIMS as compared to several existing methods is established. While 12 out of 15 selected miRNAs by FMIMS corroborate with those of biological investigations, three of them viz., "hsa-miR-519," "hsa-miR-431" and "hsa-miR-320c" are possible novel predictions for renal cancer, lung cancer and melanoma, respectively. The selected miRNAs are found to be involved in disease-specific pathways by targeting various genes. The method is also able to detect the responsible miRNAs even at the primary stage of cancer. The related code is available at http://www.jayanta.droppages.com/FMIMS.html . PMID:26264058
On the feature selection criterion based on an approximation of multidimensional mutual information.
Balagani, Kiran S; Phoha, Vir V
2010-07-01
We derive the feature selection criterion presented in [CHECK END OF SENTENCE] and [CHECK END OF SENTENCE] from the multidimensional mutual information between features and the class. Our derivation: 1) specifies and validates the lower-order dependency assumptions of the criterion and 2) mathematically justifies the utility of the criterion by relating it to Bayes classification error. PMID:20489237
Calculating mutual information for spike trains and other data with distances but no coordinates
Houghton, Conor
2015-01-01
Many important data types, such as the spike trains recorded from neurons in typical electrophysiological experiments, have a natural notion of distance or similarity between data points, even though there is no obvious coordinate system. Here, a simple Kozachenko–Leonenko estimator is derived for calculating the mutual information between datasets of this type. PMID:26064650
NASA Astrophysics Data System (ADS)
Zhang, Haihong; Guan, Cuntai
2010-10-01
This paper addresses an important issue in a self-paced brain-computer interface (BCI): constructing subject-specific continuous control signal. To this end, we propose an alternative to the conventional regression/classification-based mechanism for building the transformation from EEG features into a univariate control signal. Based on information theory, the mechanism formulates the optimum transformation as maximizing the mutual information between the control signal and the mental state. We introduce a non-parametric mutual information estimate for general output distribution, and then develop a gradient-based algorithm to optimize the transformation using training data. We conduct an offline simulation study using motor imagery data from the BCI Competition IV Data Set I. The results show that the learning algorithm converged quickly, and the proposed method yielded significantly higher BCI performance than the conventional mechanism.
Successful network inference from time-series data using mutual information rate
NASA Astrophysics Data System (ADS)
Bianco-Martinez, E.; Rubido, N.; Antonopoulos, Ch. G.; Baptista, M. S.
2016-04-01
This work uses an information-based methodology to infer the connectivity of complex systems from observed time-series data. We first derive analytically an expression for the Mutual Information Rate (MIR), namely, the amount of information exchanged per unit of time, that can be used to estimate the MIR between two finite-length low-resolution noisy time-series, and then apply it after a proper normalization for the identification of the connectivity structure of small networks of interacting dynamical systems. In particular, we show that our methodology successfully infers the connectivity for heterogeneous networks, different time-series lengths or coupling strengths, and even in the presence of additive noise. Finally, we show that our methodology based on MIR successfully infers the connectivity of networks composed of nodes with different time-scale dynamics, where inference based on Mutual Information fails.
Successful network inference from time-series data using mutual information rate.
Bianco-Martinez, E; Rubido, N; Antonopoulos, Ch G; Baptista, M S
2016-04-01
This work uses an information-based methodology to infer the connectivity of complex systems from observed time-series data. We first derive analytically an expression for the Mutual Information Rate (MIR), namely, the amount of information exchanged per unit of time, that can be used to estimate the MIR between two finite-length low-resolution noisy time-series, and then apply it after a proper normalization for the identification of the connectivity structure of small networks of interacting dynamical systems. In particular, we show that our methodology successfully infers the connectivity for heterogeneous networks, different time-series lengths or coupling strengths, and even in the presence of additive noise. Finally, we show that our methodology based on MIR successfully infers the connectivity of networks composed of nodes with different time-scale dynamics, where inference based on Mutual Information fails. PMID:27131481
Advanced algorithms for information science
Argo, P.; Brislawn, C.; Fitzgerald, T.J.; Kelley, B.; Kim, W.H.; Mazieres, B.; Roeder, H.; Strottman, D.
1998-12-31
This is the final report of a one-year, Laboratory Directed Research and Development (LDRD) project at Los Alamos National Laboratory (LANL). In a modern information-controlled society the importance of fast computational algorithms facilitating data compression and image analysis cannot be overemphasized. Feature extraction and pattern recognition are key to many LANL projects and the same types of dimensionality reduction and compression used in source coding are also applicable to image understanding. The authors have begun developing wavelet coding which decomposes data into different length-scale and frequency bands. New transform-based source-coding techniques offer potential for achieving better, combined source-channel coding performance by using joint-optimization techniques. They initiated work on a system that compresses the video stream in real time, and which also takes the additional step of analyzing the video stream concurrently. By using object-based compression schemes (where an object is an identifiable feature of the video signal, repeatable in time or space), they believe that the analysis is directly related to the efficiency of the compression.
On conclusive eavesdropping and measures of mutual information in quantum key distribution
NASA Astrophysics Data System (ADS)
Rastegin, Alexey E.
2016-03-01
We address the question of quantifying eavesdropper's information gain in an individual attack on systems of quantum key distribution. It is connected with the concept of conclusive eavesdropping introduced by Brandt. Using the BB84 protocol, we examine the problem of estimating a performance of conclusive entangling probe. The question of interest depends on the choice of a quantitative measure of eavesdropper's information about the error-free sifted bits. The Fuchs-Peres-Brandt probe realizes a very powerful individual attack on the BB84 scheme. In the usual formulation, Eve utilizes the Helstrom scheme in distinguishing between the two output probe states. In conclusive eavesdropping, the unambiguous discrimination is used. Comparing these two versions allows to demonstrate serious distinctions between widely used quantifiers of mutual information. In particular, the so-called Rényi mutual information does not seem to be a completely legitimate measure of an amount of mutual information. It is brightly emphasized with the example of conclusive eavesdropping.
Comparison of co-expression measures: mutual information, correlation, and model based indices
2012-01-01
Background Co-expression measures are often used to define networks among genes. Mutual information (MI) is often used as a generalized correlation measure. It is not clear how much MI adds beyond standard (robust) correlation measures or regression model based association measures. Further, it is important to assess what transformations of these and other co-expression measures lead to biologically meaningful modules (clusters of genes). Results We provide a comprehensive comparison between mutual information and several correlation measures in 8 empirical data sets and in simulations. We also study different approaches for transforming an adjacency matrix, e.g. using the topological overlap measure. Overall, we confirm close relationships between MI and correlation in all data sets which reflects the fact that most gene pairs satisfy linear or monotonic relationships. We discuss rare situations when the two measures disagree. We also compare correlation and MI based approaches when it comes to defining co-expression network modules. We show that a robust measure of correlation (the biweight midcorrelation transformed via the topological overlap transformation) leads to modules that are superior to MI based modules and maximal information coefficient (MIC) based modules in terms of gene ontology enrichment. We present a function that relates correlation to mutual information which can be used to approximate the mutual information from the corresponding correlation coefficient. We propose the use of polynomial or spline regression models as an alternative to MI for capturing non-linear relationships between quantitative variables. Conclusion The biweight midcorrelation outperforms MI in terms of elucidating gene pairwise relationships. Coupled with the topological overlap matrix transformation, it often leads to more significantly enriched co-expression modules. Spline and polynomial networks form attractive alternatives to MI in case of non-linear relationships
Feature selection of fMRI data based on normalized mutual information and fisher discriminant ratio.
Wang, Yanbin; Ji, Junzhong; Liang, Peipeng
2016-03-17
Pattern classification has been increasingly used in functional magnetic resonance imaging (fMRI) data analysis. However, the classification performance is restricted by the high dimensional property and noises of the fMRI data. In this paper, a new feature selection method (named as "NMI-F") was proposed by sequentially combining the normalized mutual information (NMI) and fisher discriminant ratio. In NMI-F, the normalized mutual information was firstly used to evaluate the relationships between features, and fisher discriminant ratio was then applied to calculate the importance of each feature involved. Two fMRI datasets (task-related and resting state) were used to test the proposed method. It was found that classification base on the NMI-F method could differentiate the brain cognitive and disease states effectively, and the proposed NMI-F method was prior to the other related methods. The current results also have implications to the future studies. PMID:27257882
Artifact reduction in mutual-information-based CT-MR image registration
NASA Astrophysics Data System (ADS)
Wei, Mingxiu; Liu, Jundong; Liu, Junhong
2004-05-01
Abstract Mutual information (MI) is currently the most popular match metric in handling the registration problem for multi modality images. However, interpolation artifacts impose deteriorating effects to the accuracy and robustness of MI-based methods. This paper analyzes the generation mechanism of the artifacts inherent in linear partial volume interpolation (PVI) and shows that the mutual information resulted from PVI is a convex function within each voxel grid. We conclude that the generation of the artifacts is due to two facts: 1) linear interpolation causes the histogram bin values to change at a synchronized pace; 2) entropy computation function Σxlgx is convex. As a remedy we propose to use non-uniform interpolation functions as the interpolation kernels in estimating the joint histogram. Cubic B-splin and Gaussian interpolators are compared and we demonstrate the improvements via experiments on misalignments between CT/MR brain scans.
Mutual information and self-control of a fully-connected low-activity neural network
NASA Astrophysics Data System (ADS)
Bollé, D.; Carreta, D. Dominguez
2000-11-01
A self-control mechanism for the dynamics of a three-state fully connected neural network is studied through the introduction of a time-dependent threshold. The self-adapting threshold is a function of both the neural and the pattern activity in the network. The time evolution of the order parameters is obtained on the basis of a recently developed dynamical recursive scheme. In the limit of low activity the mutual information is shown to be the relevant parameter in order to determine the retrieval quality. Due to self-control an improvement of this mutual information content as well as an increase of the storage capacity and an enlargement of the basins of attraction are found. These results are compared with numerical simulations.
Zhang, Xiujun; Zhao, Juan; Hao, Jin-Kao; Zhao, Xing-Ming; Chen, Luonan
2015-03-11
Mutual information (MI), a quantity describing the nonlinear dependence between two random variables, has been widely used to construct gene regulatory networks (GRNs). Despite its good performance, MI cannot separate the direct regulations from indirect ones among genes. Although the conditional mutual information (CMI) is able to identify the direct regulations, it generally underestimates the regulation strength, i.e. it may result in false negatives when inferring gene regulations. In this work, to overcome the problems, we propose a novel concept, namely conditional mutual inclusive information (CMI2), to describe the regulations between genes. Furthermore, with CMI2, we develop a new approach, namely CMI2NI (CMI2-based network inference), for reverse-engineering GRNs. In CMI2NI, CMI2 is used to quantify the mutual information between two genes given a third one through calculating the Kullback-Leibler divergence between the postulated distributions of including and excluding the edge between the two genes. The benchmark results on the GRNs from DREAM challenge as well as the SOS DNA repair network in Escherichia coli demonstrate the superior performance of CMI2NI. Specifically, even for gene expression data with small sample size, CMI2NI can not only infer the correct topology of the regulation networks but also accurately quantify the regulation strength between genes. As a case study, CMI2NI was also used to reconstruct cancer-specific GRNs using gene expression data from The Cancer Genome Atlas (TCGA). CMI2NI is freely accessible at http://www.comp-sysbio.org/cmi2ni. PMID:25539927
Zhang, Xiujun; Zhao, Juan; Hao, Jin-Kao; Zhao, Xing-Ming; Chen, Luonan
2015-01-01
Mutual information (MI), a quantity describing the nonlinear dependence between two random variables, has been widely used to construct gene regulatory networks (GRNs). Despite its good performance, MI cannot separate the direct regulations from indirect ones among genes. Although the conditional mutual information (CMI) is able to identify the direct regulations, it generally underestimates the regulation strength, i.e. it may result in false negatives when inferring gene regulations. In this work, to overcome the problems, we propose a novel concept, namely conditional mutual inclusive information (CMI2), to describe the regulations between genes. Furthermore, with CMI2, we develop a new approach, namely CMI2NI (CMI2-based network inference), for reverse-engineering GRNs. In CMI2NI, CMI2 is used to quantify the mutual information between two genes given a third one through calculating the Kullback–Leibler divergence between the postulated distributions of including and excluding the edge between the two genes. The benchmark results on the GRNs from DREAM challenge as well as the SOS DNA repair network in Escherichia coli demonstrate the superior performance of CMI2NI. Specifically, even for gene expression data with small sample size, CMI2NI can not only infer the correct topology of the regulation networks but also accurately quantify the regulation strength between genes. As a case study, CMI2NI was also used to reconstruct cancer-specific GRNs using gene expression data from The Cancer Genome Atlas (TCGA). CMI2NI is freely accessible at http://www.comp-sysbio.org/cmi2ni. PMID:25539927
Four-state quantum key distribution exploiting maximum mutual information measurement strategy
NASA Astrophysics Data System (ADS)
Chen, Dong-Xu; Zhang, Pei; Li, Hong-Rong; Gao, Hong; Li, Fu-Li
2016-02-01
We propose a four-state quantum key distribution (QKD) scheme using generalized measurement of nonorthogonal states, the maximum mutual information measurement strategy. Then, we analyze the eavesdropping process in intercept-resend and photon number splitting attack scenes. Our analysis shows that in the intercept-resend and photon number splitting attack eavesdropping scenes, our scheme is more secure than BB84 protocol and has higher key generation rate which may be applied to high-density QKD.
NASA Astrophysics Data System (ADS)
Pires, C. L.
2013-12-01
Principal components (PCs) of the low-frequency variability have zero cross correlation by construction but they are not statistically independent. Their degree of dependency is assessed through the Shannon mutual information (MI). PCs were computed here both for: 1) the monthly running means of the stream functions of a one million days run of a T63, 3level, perpetual winter forced, quasi-geostrophic (QG3) model and 2) the annual running means of the SST from GISS 1880-2012 data. One computes both the dyadic MI: I(X,Y) and triadic MI: I(X,Y,Z) among arbitrary PCs X,Y,Z (rotated or not) by using a kernel-based MI estimation method applied to previously Gaussianized marginal variables obtained by Gaussian anamorphosis thus making estimation more resistant to outliers. Non-vanishing MI comes from the non-Gaussianity of the full PDF of the state-vector of retained PCs. Statistically significant non-Gaussian dyadic MI appears between leading PC-pairs, both for the QG3 model run (projecting on planetary-slow scales) and for GISS data where some nonlinear correlations are emphasized between Pacific and Atlantic SST modes. We propose an iterative optimization algorithm looking for uncorrelated variables X, Y, Z, (obtained from orthogonal projections), taken from a multivariate space of N PCs (N≥3), which maximize I(X,Y,Z), i.e. their triadic non-Gaussian interaction. It also maximizes the joint negentropy leading to the presence of relevant non-linear correlations across the three linearly uncorrelated variables. This is solved through an iterative optimization method by maximizing a positive contrast function (e.g. the squared expectation E(XYZ)2 ), vanishing under Gaussian conditions. In order to understand the origin of a statistically significant positive mutual information I(X,Y,Z)>0, one decomposes it into a dyadic term: I2(X,Y,Z)≡I(X,Y)+I(X,Z)+I(Y,Z), vanishing iff X,Y,Z are pair-wised independent and into a triadic term, the so called interactivity term: It(X
Spatially weighted mutual information image registration for image guided radiation therapy
Park, Samuel B.; Rhee, Frank C.; Monroe, James I.; Sohn, Jason W.
2010-09-15
Purpose: To develop a new metric for image registration that incorporates the (sub)pixelwise differential importance along spatial location and to demonstrate its application for image guided radiation therapy (IGRT). Methods: It is well known that rigid-body image registration with mutual information is dependent on the size and location of the image subset on which the alignment analysis is based [the designated region of interest (ROI)]. Therefore, careful review and manual adjustments of the resulting registration are frequently necessary. Although there were some investigations of weighted mutual information (WMI), these efforts could not apply the differential importance to a particular spatial location since WMI only applies the weight to the joint histogram space. The authors developed the spatially weighted mutual information (SWMI) metric by incorporating an adaptable weight function with spatial localization into mutual information. SWMI enables the user to apply the selected transform to medically ''important'' areas such as tumors and critical structures, so SWMI is neither dominated by, nor neglects the neighboring structures. Since SWMI can be utilized with any weight function form, the authors presented two examples of weight functions for IGRT application: A Gaussian-shaped weight function (GW) applied to a user-defined location and a structures-of-interest (SOI) based weight function. An image registration example using a synthesized 2D image is presented to illustrate the efficacy of SWMI. The convergence and feasibility of the registration method as applied to clinical imaging is illustrated by fusing a prostate treatment planning CT with a clinical cone beam CT (CBCT) image set acquired for patient alignment. Forty-one trials are run to test the speed of convergence. The authors also applied SWMI registration using two types of weight functions to two head and neck cases and a prostate case with clinically acquired CBCT/MVCT image sets. The
Time-Delayed Mutual Information of the Phase as a Measure of Functional Connectivity
Wilmer, Andreas; de Lussanet, Marc; Lappe, Markus
2012-01-01
We propose a time-delayed mutual information of the phase for detecting nonlinear synchronization in electrophysiological data such as MEG. Palus already introduced the mutual information as a measure of synchronization [1]. To obtain estimates on small data-sets as reliably as possible, we adopt the numerical implementation as proposed by Kraskov and colleagues [2]. An embedding with a parametric time-delay allows a reconstruction of arbitrary nonstationary connective structures – so-called connectivity patterns – in a wide class of systems such as coupled oscillatory or even purely stochastic driven processes [3]. By using this method we do not need to make any assumptions about coupling directions, delay times, temporal dynamics, nonlinearities or underlying mechanisms. For verifying and refining the methods we generate synthetic data-sets by a mutual amplitude coupled network of Rössler oscillators with an a-priori known connective structure. This network is modified in such a way, that the power-spectrum forms a power law, which is also observed in electrophysiological recordings. The functional connectivity measure is tested on robustness to additive uncorrelated noise and in discrimination of linear mixed input data. For the latter issue a suitable de-correlation technique is applied. Furthermore, the compatibility to inverse methods for a source reconstruction in MEG such as beamforming techniques is controlled by dedicated dipole simulations. Finally, the method is applied on an experimental MEG recording. PMID:23028571
Using Mutual Information Criterion to Design an Efficient Phoneme Set for Chinese Speech Recognition
NASA Astrophysics Data System (ADS)
Zhang, Jin-Song; Hu, Xin-Hui; Nakamura, Satoshi
Chinese is a representative tonal language, and it has been an attractive topic of how to process tone information in the state-of-the-art large vocabulary speech recognition system. This paper presents a novel way to derive an efficient phoneme set of tone-dependent units to build a recognition system, by iteratively merging a pair of tone-dependent units according to the principle of minimal loss of the Mutual Information (MI). The mutual information is measured between the word tokens and their phoneme transcriptions in a training text corpus, based on the system lexical and language model. The approach has a capability to keep discriminative tonal (and phoneme) contrasts that are most helpful for disambiguating homophone words due to lack of tones, and merge those tonal (and phoneme) contrasts that are not important for word disambiguation for the recognition task. This enables a flexible selection of phoneme set according to a balance between the MI information amount and the number of phonemes. We applied the method to traditional phoneme set of Initial/Finals, and derived several phoneme sets with different number of units. Speech recognition experiments using the derived sets showed its effectiveness.
Query-answering algorithms for information agents
Levy, A.Y.; Rajaraman, A.; Ordille, J.J.
1996-12-31
We describe the architecture and query-answering algorithms used in the Information Manifold, an implemented information gathering system that provides uniform access to structured information sources on the World-Wide Web. Our architecture provides an expressive language for describing information sources, which makes it easy to add new sources and to model the fine-grained distinctions between their contents. The query-answering algorithm guarantees that the descriptions of the sources are exploited to access only sources that are relevant to a given query. Accessing only relevant sources is crucial to scale up such a system to large numbers of sources. In addition, our algorithm can exploit run-time information to further prune information sources and to reduce the cost of query planning.
Analysis of phylogenetic signal in protostomial intron patterns using Mutual Information.
Hill, Natascha; Leow, Alexander; Bleidorn, Christoph; Groth, Detlef; Tiedemann, Ralph; Selbig, Joachim; Hartmann, Stefanie
2013-06-01
Many deep evolutionary divergences still remain unresolved, such as those among major taxa of the Lophotrochozoa. As alternative phylogenetic markers, the intron-exon structure of eukaryotic genomes and the patterns of absence and presence of spliceosomal introns appear to be promising. However, given the potential homoplasy of intron presence, the phylogenetic analysis of this data using standard evolutionary approaches has remained a challenge. Here, we used Mutual Information (MI) to estimate the phylogeny of Protostomia using gene structure data, and we compared these results with those obtained with Dollo Parsimony. Using full genome sequences from nine Metazoa, we identified 447 groups of orthologous sequences with 21,732 introns in 4,870 unique intron positions. We determined the shared absence and presence of introns in the corresponding sequence alignments and have made this data available in "IntronBase", a web-accessible and downloadable SQLite database. Our results obtained using Dollo Parsimony are obviously misled through systematic errors that arise from multiple intron loss events, but extensive filtering of data improved the quality of the estimated phylogenies. Mutual Information, in contrast, performs better with larger datasets, but at the same time it requires a complete data set, which is difficult to obtain for orthologs from a large number of taxa. Nevertheless, Mutual Information-based distances proved to be useful in analyzing this kind of data, also because the estimation of MI-based distances is independent of evolutionary models and therefore no pre-definitions of ancestral and derived character states are necessary. PMID:23248024
2015-01-01
Background The rapid advances in genome sequencing technologies have resulted in an unprecedented number of genome variations being discovered in humans. However, there has been very limited coverage of interpretation of the personal genome sequencing data in terms of diseases. Methods In this paper we present the first computational analysis scheme for interpreting personal genome data by simultaneously considering the functional impact of damaging variants and curated disease-gene association data. This method is based on mutual information as a measure of the relative closeness between the personal genome and diseases. We hypothesize that a higher mutual information score implies that the personal genome is more susceptible to a particular disease than other diseases. Results The method was applied to the sequencing data of 50 acute myeloid leukemia (AML) patients in The Cancer Genome Atlas. The utility of associations between a disease and the personal genome was explored using data of healthy (control) people obtained from the 1000 Genomes Project. The ranks of the disease terms in the AML patient group were compared with those in the healthy control group using "Leukemia, Myeloid, Acute" (C04.557.337.539.550) as the corresponding MeSH disease term. The mutual information rank of the disease term was substantially higher in the AML patient group than in the healthy control group, which demonstrates that the proposed methodology can be successfully applied to infer associations between the personal genome and diseases. Conclusions Overall, the area under the receiver operating characteristics curve was significantly larger for the AML patient data than for the healthy controls. This methodology could contribute to consequential discoveries and explanations for mining personal genome sequencing data in terms of diseases, and have versatility with respect to genomic-based knowledge such as drug-gene and environmental-factor-gene interactions. PMID:26045178
A Method for Evaluating Tuning Functions of Single Neurons based on Mutual Information Maximization
NASA Astrophysics Data System (ADS)
Brostek, Lukas; Eggert, Thomas; Ono, Seiji; Mustari, Michael J.; Büttner, Ulrich; Glasauer, Stefan
2011-03-01
We introduce a novel approach for evaluation of neuronal tuning functions, which can be expressed by the conditional probability of observing a spike given any combination of independent variables. This probability can be estimated out of experimentally available data. By maximizing the mutual information between the probability distribution of the spike occurrence and that of the variables, the dependence of the spike on the input variables is maximized as well. We used this method to analyze the dependence of neuronal activity in cortical area MSTd on signals related to movement of the eye and retinal image movement.
Hybrid Online Mobile Laser Scanner Calibration Through Image Alignment by Mutual Information
NASA Astrophysics Data System (ADS)
Miled, Mourad; Soheilian, Bahman; Habets, Emmanuel; Vallet, Bruno
2016-06-01
This paper proposes an hybrid online calibration method for a laser scanner mounted on a mobile platform also equipped with an imaging system. The method relies on finding the calibration parameters that best align the acquired points cloud to the images. The quality of this intermodal alignment is measured by Mutual information between image luminance and points reflectance. The main advantage and motivation is ensuring pixel accurate alignment of images and point clouds acquired simultaneously, but it is also much more flexible than traditional laser calibration methods.
Weighted mutual information analysis substantially improves domain-based functional network models
Shim, Jung Eun; Lee, Insuk
2016-01-01
Motivation: Functional protein–protein interaction (PPI) networks elucidate molecular pathways underlying complex phenotypes, including those of human diseases. Extrapolation of domain–domain interactions (DDIs) from known PPIs is a major domain-based method for inferring functional PPI networks. However, the protein domain is a functional unit of the protein. Therefore, we should be able to effectively infer functional interactions between proteins based on the co-occurrence of domains. Results: Here, we present a method for inferring accurate functional PPIs based on the similarity of domain composition between proteins by weighted mutual information (MI) that assigned different weights to the domains based on their genome-wide frequencies. Weighted MI outperforms other domain-based network inference methods and is highly predictive for pathways as well as phenotypes. A genome-scale human functional network determined by our method reveals numerous communities that are significantly associated with known pathways and diseases. Domain-based functional networks may, therefore, have potential applications in mapping domain-to-pathway or domain-to-phenotype associations. Availability and Implementation: Source code for calculating weighted mutual information based on the domain profile matrix is available from www.netbiolab.org/w/WMI. Contact: Insuklee@yonsei.ac.kr Supplementary information: Supplementary data are available at Bioinformatics online. PMID:27207946
Wang, X-D; Qi, Y-X; Jiang, Z-L
2011-03-01
Many methods had been developed on inferring transcriptional network from gene expression. However, it is still necessary to design new method that discloses more detailed and exact network information. Using network-assisted regression, the authors combined the averaged three-way mutual information (AMI3) and non-linear ordinary differential equation (ODE) model to infer the transcriptional network, and to obtain both the topological structure and the regulatory dynamics. Synthetic and experimental data were used to evaluate the performance of the above approach. In comparison with the previous methods based on mutual information, AMI3 obtained higher precision with the same sensitivity. To describe the regulatory dynamics between transcription factors and target genes, network-assisted regression and regression without network, respectively, were applied in the steady-state and time series microarray data. The results revealed that comparing with regression without network, network-assisted regression increased the precision, but decreased the fitting goodness. Then, the authors reconstructed the transcriptional network of Escherichia coli and simulated the regulatory dynamics of genes. Furthermore, the authors' approach identified potential transcription factors regulating yeast cell cycle. In conclusion, network-assisted regression, combined AMI3 and ODE model, was a more precisely to infer the topological structure and the regulatory dynamics of transcriptional network from microarray data. [Includes supplementary material]. PMID:21405197
Mazurowski, Maciej A; Lo, Joseph Y; Harrawood, Brian P; Tourassi, Georgia D
2011-10-01
Development of a computational decision aid for a new medical imaging modality typically is a long and complicated process. It consists of collecting data in the form of images and annotations, development of image processing and pattern recognition algorithms for analysis of the new images and finally testing of the resulting system. Since new imaging modalities are developed more rapidly than ever before, any effort for decreasing the time and cost of this development process could result in maximizing the benefit of the new imaging modality to patients by making the computer aids quickly available to radiologists that interpret the images. In this paper, we make a step in this direction and investigate the possibility of translating the knowledge about the detection problem from one imaging modality to another. Specifically, we present a computer-aided detection (CAD) system for mammographic masses that uses a mutual information-based template matching scheme with intelligently selected templates. We presented principles of template matching with mutual information for mammography before. In this paper, we present an implementation of those principles in a complete computer-aided detection system. The proposed system, through an automatic optimization process, chooses the most useful templates (mammographic regions of interest) using a large database of previously collected and annotated mammograms. Through this process, the knowledge about the task of detecting masses in mammograms is incorporated in the system. Then, we evaluate whether our system developed for screen-film mammograms can be successfully applied not only to other mammograms but also to digital breast tomosynthesis (DBT) reconstructed slices without adding any DBT cases for training. Our rationale is that since mutual information is known to be a robust inter-modality image similarity measure, it has high potential of transferring knowledge between modalities in the context of the mass detection
Universal behavior of the Shannon and Rényi mutual information of quantum critical chains
NASA Astrophysics Data System (ADS)
Alcaraz, F. C.; Rajabpour, M. A.
2014-08-01
We study the Shannon and Rényi mutual information (MI) in the ground state (GS) of different critical quantum spin chains. Despite the apparent basis dependence of these quantities we show the existence of some particular basis (we will call them conformal basis) whose finite-size scaling function is related to the central charge c of the underlying conformal field theory of the model. In particular, we verified that for large index n, the MI of a subsystem of size ℓ in a periodic chain with L sites behaves as c/4 n/n -1 ln[L/πsin(π/ℓL)], when the ground-state wave function is expressed in these special conformal basis. This is in agreement with recent predictions. For generic local basis, we will show that, although in some cases bnln[L/π sin(π/ℓL)] is a good fit to our numerical data, in general, there is no direct relation between bn and the central charge of the system. We will support our findings with detailed numerical calculations for the transverse field Ising model, Q =3,4 quantum Potts chain, quantum Ashkin-Teller chain, and the XXZ quantum chain. We will also present some additional results of the Shannon mutual information (n =1), for the parafermionic ZQ quantum chains with Q =5,6,7, and 8.
Shannon and Rényi mutual information in quantum critical spin chains
NASA Astrophysics Data System (ADS)
Stéphan, Jean-Marie
2014-07-01
We study the Shannon mutual information in one-dimensional critical spin chains, following a recent conjecture [Alcaraz and Rajabpour, Phys. Rev. Lett. 111, 017201 (2013), 10.1103/PhysRevLett.111.017201], as well as Rényi generalizations of it. We combine conformal field theory (CFT) arguments with numerical computations in lattice discretizations with central charge c =1 and c =1/2. For a periodic system of length L cut into two parts of length ℓ and L -ℓ, all our results agree with the general shape dependence In(ℓ,L)=(bn/4)ln(L/π sin π/ℓL), where bn is a universal coefficient. For the free boson CFT we show from general arguments that bn=c =1. At c =1/2 we conjecture a result for n >1. We perform extensive numerical computations in Ising chains to confirm this, and also find b1≃0.4801629(2), a nontrivial number which we do not understand analytically. Open chains at c =1/2 and n =1 are even more intriguing, with a shape-dependent logarithmic divergence of the Shannon mutual information.
NASA Astrophysics Data System (ADS)
Ikemoto, Shuhei; DallaLibera, Fabio; Hosoda, Koh; Ishiguro, Hiroshi
2014-10-01
Stochastic resonance (SR) is a counterintuitive phenomenon, observed in a wide variety of nonlinear systems, for which the addition of noise of opportune magnitude can improve signal detection. Tuning the noise for maximizing the SR effect is important both for artificial and biological systems. In the case of artificial systems, full exploitation of the SR effect opens the possibility of measuring otherwise unmeasurable signals. In biology, identification of possible SR maximization mechanisms is of great interest for explaining the low-energy high-sensitivity perception capabilities often observed in animals. SR maximization approaches presented in literature use knowledge on the input signal (or stimulus, in the case of living beings), and maximize the mutual information between the input and the output signal. The input signal, however, is unknown in many practical settings. To cope with this problem, this paper introduces an approximation of the input-output mutual information based on the spurious correlation among a set of redundant units. A proof of the approximation, as well as numerical examples of its application are given.
Neural sensitivity to syllable frequency and mutual information in speech perception and production.
Tremblay, Pascale; Deschamps, Isabelle; Baroni, Marco; Hasson, Uri
2016-08-01
Many factors affect our ability to decode the speech signal, including its quality, the complexity of the elements that compose it, as well as their frequency of occurrence and co-occurrence in a language. Syllable frequency effects have been described in the behavioral literature, including facilitatory effects during speech production and inhibitory effects during word recognition, but the neural mechanisms underlying these effects remain largely unknown. The objective of this study was to examine, using functional neuroimaging, the neurobiological correlates of three different distributional statistics in simple 2-syllable nonwords: the frequency of the first and second syllables, and the mutual information between the syllables. We examined these statistics during nonword perception and production using a powerful single-trial analytical approach. We found that repetition accuracy was higher for nonwords in which the frequency of the first syllable was high. In addition, brain responses to distributional statistics were widespread and almost exclusively cortical. Importantly, brain activity was modulated in a distinct manner for each statistic, with the strongest facilitatory effects associated with the frequency of the first syllable and mutual information. These findings show that distributional statistics modulate nonword perception and production. We discuss the common and unique impact of each distributional statistic on brain activity, as well as task differences. PMID:27184201
NASA Technical Reports Server (NTRS)
Wolf, David R.
2004-01-01
The topic of this paper is a hierarchy of information-like functions, here named the information correlation functions, where each function of the hierarchy may be thought of as the information between the variables it depends upon. The information correlation functions are particularly suited to the description of the emergence of complex behaviors due to many- body or many-agent processes. They are particularly well suited to the quantification of the decomposition of the information carried among a set of variables or agents, and its subsets. In more graphical language, they provide the information theoretic basis for understanding the synergistic and non-synergistic components of a system, and as such should serve as a forceful toolkit for the analysis of the complexity structure of complex many agent systems. The information correlation functions are the natural generalization to an arbitrary number of sets of variables of the sequence starting with the entropy function (one set of variables) and the mutual information function (two sets). We start by describing the traditional measures of information (entropy) and mutual information.
NASA Astrophysics Data System (ADS)
Sorensen, Julian
2008-12-01
At the heart of many ICA techniques is a nonparametric estimate of an information measure, usually via nonparametric density estimation, for example, kernel density estimation. While not as popular as kernel density estimators, orthogonal functions can be used for nonparametric density estimation (via a truncated series expansion whose coefficients are calculated from the observed data). While such estimators do not necessarily yield a valid density, which kernel density estimators do, they are faster to calculate than kernel density estimators, in particular for a modified version of Renyi's entropy of order 2. In this paper, we compare the performance of ICA using Hermite series based estimates of Shannon's and Renyi's mutual information, to that of Gaussian kernel based estimates. The comparisons also include ICA using the RADICAL estimate of Shannon's entropy and a FastICA estimate of neg-entropy.
Measuring the usefulness of hidden units in Boltzmann machines with mutual information.
Berglund, Mathias; Raiko, Tapani; Cho, Kyunghyun
2015-04-01
Restricted Boltzmann machines (RBMs) and deep Boltzmann machines (DBMs) are important models in deep learning, but it is often difficult to measure their performance in general, or measure the importance of individual hidden units in specific. We propose to use mutual information to measure the usefulness of individual hidden units in Boltzmann machines. The measure is fast to compute, and serves as an upper bound for the information the neuron can pass on, enabling detection of a particular kind of poor training results. We confirm experimentally that the proposed measure indicates how much the performance of the model drops when some of the units of an RBM are pruned away. We demonstrate the usefulness of the measure for early detection of poor training in DBMs. PMID:25318376
Whitmore, Rebecca; Crooks, Valorie A; Snyder, Jeremy
2015-09-01
This study examines the experiences of informal caregivers in medical tourism through an ethics of care lens. We conducted semi-structured interviews with 20 Canadians who had accompanied their friends or family members abroad for surgery, asking questions that dealt with their experiences prior to, during and after travel. Thematic analysis revealed three themes central to an ethics of care: responsibility, vulnerability and mutuality. Ethics of care theorists have highlighted how care has been historically devalued. We posit that medical tourism reproduces dominant narratives about care in a novel care landscape. Informal care goes unaccounted for by the industry, as it occurs in largely private spaces at a geographic distance from the home countries of medical tourists. PMID:26313855
Using Mutual Information to capture Major Concerns of Postural Control in a Tossing activity
Gazula, Harshvardhan; Chang, Chien Chi; Lu, Ming-Lun; Hsiang, Simon M.
2015-01-01
Human body motion for load-tossing activity was partitioned into three phases using four critical events based on the load position viz. lift-off, closest to body, peak and release. For each phase, three objective functions values, viz. mobilization, stabilization and muscular torque utilization, used to control the motion patterns, were then calculated. We hypothesize that the relationships between different objective functions can be extracted using information theory. The kinematic data obtained with 36 treatment combinations (2 tossing distances, 2 tossing heights, 3 weights, and 3 target clearances) was used to estimate the mutual information between each pair of objective functions and construct Chow-Liu trees. Results from this research indicate that there was no dominant concern in the first two phases of the activity; however, torque utilization and mobilization were found to be important factors in the third phase of the load tossing activity. PMID:25680297
Mutual Information in the Air Quality Monitoring Network of Bogota - Colombia
NASA Astrophysics Data System (ADS)
Guerrero, O. J.; Jimenez-Pizarro, R.
2012-12-01
Large urban areas in the developing world are characterized by high population density and a great variety of activities responsible for emission of trace gases and particulate matter to the atmosphere. In general, these pollutants are unevenly distributed over cities according to the location of sources, meteorological variability and geographical features. Urban air quality monitoring networks are primarily designed to protect public health. The meteorological and air quality information gathered by monitoring networks can also be used to understand pollutant sources, sinks, and dispersion processes and to assess the spatial coverage of the network itself. Several statistical and numerical simulation methods allow for the identification of the domain that influences observations at each of the stations, i.e, the zone and respective population truly covered by the measurements. We focused on Bogota, Colombia, a dense city of approximately 9.6 million inhabitants in its metropolitan area. We analyzed the measurements obtained by the Bogotá Air Quality Monitoring Network (RMCAB) between the years 1997 and 2010 for TSP, PM10, CO, NOx and O3. RMCAB is composed of 16 stations, 13 of which are fixed and measure both atmospheric pollutants and meteorological variables. The method applied consisted of a statistical approach based on the mutual information that each station shares with its complement, i.e. the set formed by the other stations of the network. In order to improve our understanding and interpretation of the results, virtual data created for selected receptors along a simple modeled Gaussian plume spreading throughout Bogotá was analyzed. In this Gaussian model, we accounted for the prevailing weather conditions of this city and for different emission features under which the pollutants are emitted. The spatial location of the monitoring stations and emission sources, and the quality of the measurements are relevant factors when assessing the mutual
A frequency-resolved mutual information rate and its application to neural systems.
Bernardi, Davide; Lindner, Benjamin
2015-03-01
The encoding and processing of time-dependent signals into sequences of action potentials of sensory neurons is still a challenging theoretical problem. Although, with some effort, it is possible to quantify the flow of information in the model-free framework of Shannon's information theory, this yields just a single number, the mutual information rate. This rate does not indicate which aspects of the stimulus are encoded. Several studies have identified mechanisms at the cellular and network level leading to low- or high-pass filtering of information, i.e., the selective coding of slow or fast stimulus components. However, these findings rely on an approximation, specifically, on the qualitative behavior of the coherence function, an approximate frequency-resolved measure of information flow, whose quality is generally unknown. Here, we develop an assumption-free method to measure a frequency-resolved information rate about a time-dependent Gaussian stimulus. We demonstrate its application for three paradigmatic descriptions of neural firing: an inhomogeneous Poisson process that carries a signal in its instantaneous firing rate; an integrator neuron (stochastic integrate-and-fire model) driven by a time-dependent stimulus; and the synchronous spikes fired by two commonly driven integrator neurons. In agreement with previous coherence-based estimates, we find that Poisson and integrate-and-fire neurons are broadband and low-pass filters of information, respectively. The band-pass information filtering observed in the coherence of synchronous spikes is confirmed by our frequency-resolved information measure in some but not all parameter configurations. Our results also explicitly show how the response-response coherence can fail as an upper bound on the information rate. PMID:25475346
Entanglement and mutual information in two-dimensional nonrelativistic field theories
NASA Astrophysics Data System (ADS)
Hosseini, Seyed Morteza; Véliz-Osorio, Álvaro
2016-01-01
We carry out a systematic study of entanglement entropy in nonrelativistic conformal field theories via holographic techniques. After a discussion of recent results concerning Galilean conformal field theories, we deduce a novel expression for the entanglement entropy of (1 +1 )-dimensional Lifshitz field theories—this is done both at zero and finite temperature. Based on these results, we pose a conjecture for the anomaly coefficient of a Lifshitz field theory dual to new massive gravity. It is found that the Lifshitz entanglement entropy at finite temperature displays a striking similarity with that corresponding to a flat space cosmology in three dimensions. We claim that this structure is an inherent feature of the entanglement entropy for nonrelativistic conformal field theories. We finish by exploring the behavior of the mutual information for such theories.
Excited-state entanglement and thermal mutual information in random spin chains
NASA Astrophysics Data System (ADS)
Huang, Yichen; Moore, Joel E.
2014-12-01
Entanglement properties of excited eigenstates (or of thermal mixed states) are difficult to study with conventional analytical methods. We approach this problem for random spin chains using a recently developed real-space renormalization group technique for excited states ("RSRG-X"). For the random XX and quantum Ising chains, which have logarithmic divergences in the entanglement entropy of their (infinite-randomness) critical ground states, we show that the entanglement entropy of excited eigenstates retains a logarithmic divergence while the mutual information of thermal mixed states does not. However, in the XX case the coefficient of the logarithmic divergence extends from the universal ground-state value to a universal interval due to the degeneracy of excited eigenstates. These models are noninteracting in the sense of having free-fermion representations, allowing strong numerical checks of our analytical predictions.
NASA Astrophysics Data System (ADS)
Sameni, R.; Vrins, F.; Parmentier, F.; Hérail, C.; Vigneron, V.; Verleysen, M.; Jutten, C.; Shamsollahi, M. B.
2006-11-01
Blind source separation (BSS) techniques have revealed to be promising approaches for the noninvasive extraction of fetal cardiac signals from maternal abdominal recordings. From previous studies, it is now believed that a carefully selected array of electrodes well-placed over the abdomen of a pregnant woman contains the required `information' for BSS, to extract the complete fetal components. Based on this idea, previous works have involved array recording systems and sensor selection strategies based on the Mutual Information (MI) criterion. In this paper the previous works have been extended, by considering the 3-dimensional aspects of the cardiac electrical activity. The proposed method has been tested on simulated and real maternal abdominal recordings. The results show that the new sensor selection strategy together with the MI criterion, can be effectively used to select the channels containing the most `information' concerning the fetal ECG components from an array of 72 recordings. The method is hence believed to be useful for the selection of the most informative channels in online applications, considering the different fetal positions and movements.
NASA Astrophysics Data System (ADS)
Luo, Xi-Liu; Wang, Jiang; Han, Chun-Xiao; Deng, Bin; Wei, Xi-Le; Bian, Hong-Rui
2012-02-01
As a convenient approach to the characterization of cerebral cortex electrical information, electroencephalograph (EEG) has potential clinical application in monitoring the acupuncture effects. In this paper, a method composed of the mutual information method and Lempel—Ziv complexity method (MILZC) is proposed to investigate the effects of acupuncture on the complexity of information exchanges between different brain regions based on EEGs. In the experiments, eight subjects are manually acupunctured at ‘Zusanli’ acupuncture point (ST-36) with different frequencies (i.e., 50, 100, 150, and 200 times/min) and the EEGs are recorded simultaneously. First, MILZC values are compared in general. Then average brain connections are used to quantify the effectiveness of acupuncture under the above four frequencies. Finally, significance index P values are used to study the spatiality of the acupuncture effect on the brain. Three main findings are obtained: (i) MILZC values increase during the acupuncture; (ii) manual acupunctures (MAs) with 100 times/min and 150 times/min are more effective than with 50 times/min and 200 times/min; (iii) contralateral hemisphere activation is more prominent than ipsilateral hemisphere's. All these findings suggest that acupuncture contributes to the increase of brain information exchange complexity and the MILZC method can successfully describe these changes.
NASA Astrophysics Data System (ADS)
Nielsen, O. F.; Ploug, C.; Mendoza, J. A.; Martínez, K.
2009-05-01
The need for increaseding accuracy and reduced ambiguities in the inversion results has resulted in focus on the development of more advanced inversion methods of geophysical data. Over the past few years more advanced inversion techniques have been developed to improve the results. Real 3D-inversion is time consuming and therefore often not the best solution in a cost-efficient perspective. This has motivated the development of 3D constrained inversions, where 1D-models are constrained in 3D, also known as a Spatial Constrained Inversion (SCI). Moreover, inversion of several different data types in one inversion has been developed, known as Mutually Constrained Inversion (MCI). In this paper a presentation of a Spatial Mutually Constrained Inversion method (SMCI) is given. This method allows 1D-inversion applied to different geophysical datasets and geological information constrained in 3D. Application of two or more types of geophysical methods in the inversion has proved to reduce the equivalence problem and to increase the resolution in the inversion results. The use of geological information from borehole data or digital geological models can be integrated in the inversion. In the SMCI, a 1D inversion code is used to model soundings that are constrained in three dimensions according to their relative position in space. This solution enhances the accuracy of the inversion and produces distinct layers thicknesses and resistivities. It is very efficient in the mapping of a layered geology but still also capable of mapping layer discontinuities that are, in many cases, related to fracturing and faulting or due to valley fills. Geological information may be included in the inversion directly or used only to form a starting model for the individual soundings in the inversion. In order to show the effectiveness of the method, examples are presented from both synthetic data and real data. The examples include DC-soundings as well as land-based and airborne TEM
Automatic registration of optical imagery with 3d lidar data using local combined mutual information
NASA Astrophysics Data System (ADS)
Parmehr, E. G.; Fraser, C. S.; Zhang, C.; Leach, J.
2013-10-01
Automatic registration of multi-sensor data is a basic step in data fusion for photogrammetric and remote sensing applications. The effectiveness of intensity-based methods such as Mutual Information (MI) for automated registration of multi-sensor image has been previously reported for medical and remote sensing applications. In this paper, a new multivariable MI approach that exploits complementary information of inherently registered LiDAR DSM and intensity data to improve the robustness of registering optical imagery and LiDAR point cloud, is presented. LiDAR DSM and intensity information has been utilised in measuring the similarity of LiDAR and optical imagery via the Combined MI. An effective histogramming technique is adopted to facilitate estimation of a 3D probability density function (pdf). In addition, a local similarity measure is introduced to decrease the complexity of optimisation at higher dimensions and computation cost. Therefore, the reliability of registration is improved due to the use of redundant observations of similarity. The performance of the proposed method for registration of satellite and aerial images with LiDAR data in urban and rural areas is experimentally evaluated and the results obtained are discussed.
Omitaomu, Olufemi A; Protopopescu, Vladimir A; Ganguly, Auroop R
2011-01-01
A new approach is developed for denoising signals using the Empirical Mode Decomposition (EMD) technique and the Information-theoretic method. The EMD technique is applied to decompose a noisy sensor signal into the so-called intrinsic mode functions (IMFs). These functions are of the same length and in the same time domain as the original signal. Therefore, the EMD technique preserves varying frequency in time. Assuming the given signal is corrupted by high-frequency Gaussian noise implies that most of the noise should be captured by the first few modes. Therefore, our proposition is to separate the modes into high-frequency and low-frequency groups. We applied an information-theoretic method, namely mutual information, to determine the cut-off for separating the modes. A denoising procedure is applied only to the high-frequency group using a shrinkage approach. Upon denoising, this group is combined with the original low-frequency group to obtain the overall denoised signal. We illustrate our approach with simulated and real-world data sets. The results are compared to two popular denoising techniques in the literature, namely discrete Fourier transform (DFT) and discrete wavelet transform (DWT). We found that our approach performs better than DWT and DFT in most cases, and comparatively to DWT in some cases in terms of: (i) mean square error, (ii) recomputed signal-to-noise ratio, and (iii) visual quality of the denoised signals.
A mutual information-based metric for evaluation of fMRI data-processing approaches.
Afshin-Pour, Babak; Soltanian-Zadeh, Hamid; Hossein-Zadeh, Gholam-Ali; Grady, Cheryl L; Strother, Stephen C
2011-05-01
We propose a novel approach for evaluating the performance of activation detection in real (experimental) datasets using a new mutual information (MI)-based metric and compare its sensitivity to several existing performance metrics in both simulated and real datasets. The proposed approach is based on measuring the approximate MI between the fMRI time-series of a validation dataset and a calculated activation map (thresholded label map or continuous map) from an independent training dataset. The MI metric is used to measure the amount of information preserved during the extraction of an activation map from experimentally related fMRI time-series. The processing method that preserves maximal information between the maps and related time-series is proposed to be superior. The results on simulation datasets for multiple analysis models are consistent with the results of ROC curves, but are shown to have lower information content than for real datasets, limiting their generalizability. In real datasets for group analyses using the general linear model (GLM; FSL4 and SPM5), we show that MI values are (1) larger for groups of 15 versus 10 subjects and (2) more sensitive measures than reproducibility (for continuous maps) or Jaccard overlap metrics (for thresholded maps). We also show that (1) for an increasing fraction of nominally active voxels, both MI and false discovery rate (FDR) increase, and (2) at a fixed FDR, GLM using FSL4 tends to extract more voxels and more information than SPM5 using the default processing techniques in each package. PMID:20533565
Iskarous, Khalil; Mooshammer, Christine; Hoole, Phil; Recasens, Daniel; Shadle, Christine H.; Saltzman, Elliot; Whalen, D. H.
2013-01-01
Coarticulation and invariance are two topics at the center of theorizing about speech production and speech perception. In this paper, a quantitative scale is proposed that places coarticulation and invariance at the two ends of the scale. This scale is based on physical information flow in the articulatory signal, and uses Information Theory, especially the concept of mutual information, to quantify these central concepts of speech research. Mutual Information measures the amount of physical information shared across phonological units. In the proposed quantitative scale, coarticulation corresponds to greater and invariance to lesser information sharing. The measurement scale is tested by data from three languages: German, Catalan, and English. The relation between the proposed scale and several existing theories of coarticulation is discussed, and implications for existing theories of speech production and perception are presented. PMID:23927125
Verification of 3d Building Models Using Mutual Information in Airborne Oblique Images
NASA Astrophysics Data System (ADS)
Nyaruhuma, A. P.; Gerke, M.; Vosselman, G.
2012-07-01
This paper describes a method for automatic verification of 3D building models using airborne oblique images. The problem being tackled is identifying buildings that are demolished or changed since the models were constructed or identifying wrong models using the images. The models verified are of CityGML LOD2 or higher since their edges are expected to coincide with actual building edges. The verification approach is based on information theory. Corresponding variables between building models and oblique images are used for deriving mutual information for individual edges, faces or whole buildings, and combined for all perspective images available for the building. The wireframe model edges are projected to images and verified using low level image features - the image pixel gradient directions. A building part is only checked against images in which it may be visible. The method has been tested with models constructed using laser points against Pictometry images that are available for most cities of Europe and may be publically viewed in the so called Birds Eye view of the Microsoft Bing Maps. Results are that nearly all buildings are correctly categorised as existing or demolished. Because we now concentrate only on roofs we also used the method to test and compare results from nadir images. This comparison made clear that especially height errors in models can be more reliably detected in oblique images because of the tilted view. Besides overall building verification, results per individual edges can be used for improving the 3D building models.
Aguilar, Daniel; Oliva, Baldo; Marino Buslje, Cristina
2012-01-01
Amino acids committed to a particular function correlate tightly along evolution and tend to form clusters in the 3D structure of the protein. Consequently, a protein can be seen as a network of co-evolving clusters of residues. The goal of this work is two-fold: first, we have combined mutual information and structural data to describe the amino acid networks within a protein and their interactions. Second, we have investigated how this information can be used to improve methods of prediction of functional residues by reducing the search space. As a main result, we found that clusters of co-evolving residues related to the catalytic site of an enzyme have distinguishable topological properties in the network. We also observed that these clusters usually evolve independently, which could be related to a fail-safe mechanism. Finally, we discovered a significant enrichment of functional residues (e.g. metal binding, susceptibility to detrimental mutations) in the clusters, which could be the foundation of new prediction tools. PMID:22848494
Aguilar, Daniel; Oliva, Baldo; Marino Buslje, Cristina
2012-01-01
Amino acids committed to a particular function correlate tightly along evolution and tend to form clusters in the 3D structure of the protein. Consequently, a protein can be seen as a network of co-evolving clusters of residues. The goal of this work is two-fold: first, we have combined mutual information and structural data to describe the amino acid networks within a protein and their interactions. Second, we have investigated how this information can be used to improve methods of prediction of functional residues by reducing the search space. As a main result, we found that clusters of co-evolving residues related to the catalytic site of an enzyme have distinguishable topological properties in the network. We also observed that these clusters usually evolve independently, which could be related to a fail-safe mechanism. Finally, we discovered a significant enrichment of functional residues (e.g. metal binding, susceptibility to detrimental mutations) in the clusters, which could be the foundation of new prediction tools. PMID:22848494
2014-01-01
Background Several methods are available for the detection of covarying positions from a multiple sequence alignment (MSA). If the MSA contains a large number of sequences, information about the proximities between residues derived from covariation maps can be sufficient to predict a protein fold. However, in many cases the structure is already known, and information on the covarying positions can be valuable to understand the protein mechanism and dynamic properties. Results In this study we have sought to determine whether a multivariate (multidimensional) extension of traditional mutual information (MI) can be an additional tool to study covariation. The performance of two multidimensional MI (mdMI) methods, designed to remove the effect of ternary/quaternary interdependencies, was tested with a set of 9 MSAs each containing <400 sequences, and was shown to be comparable to that of the newest methods based on maximum entropy/pseudolikelyhood statistical models of protein sequences. However, while all the methods tested detected a similar number of covarying pairs among the residues separated by < 8 Å in the reference X-ray structures, there was on average less than 65% overlap between the top scoring pairs detected by methods that are based on different principles. Conclusions Given the large variety of structure and evolutionary history of different proteins it is possible that a single best method to detect covariation in all proteins does not exist, and that for each protein family the best information can be derived by merging/comparing results obtained with different methods. This approach may be particularly valuable in those cases in which the size of the MSA is small or the quality of the alignment is low, leading to significant differences in the pairs detected by different methods. PMID:24886131
Registration of 2D to 3D joint images using phase-based mutual information
NASA Astrophysics Data System (ADS)
Dalvi, Rupin; Abugharbieh, Rafeef; Pickering, Mark; Scarvell, Jennie; Smith, Paul
2007-03-01
Registration of two dimensional to three dimensional orthopaedic medical image data has important applications particularly in the area of image guided surgery and sports medicine. Fluoroscopy to computer tomography (CT) registration is an important case, wherein digitally reconstructed radiographs derived from the CT data are registered to the fluoroscopy data. Traditional registration metrics such as intensity-based mutual information (MI) typically work well but often suffer from gross misregistration errors when the image to be registered contains a partial view of the anatomy visible in the target image. Phase-based MI provides a robust alternative similarity measure which, in addition to possessing the general robustness and noise immunity that MI provides, also employs local phase information in the registration process which makes it less susceptible to the aforementioned errors. In this paper, we propose using the complex wavelet transform for computing image phase information and incorporating that into a phase-based MI measure for image registration. Tests on a CT volume and 6 fluoroscopy images of the knee are presented. The femur and the tibia in the CT volume were individually registered to the fluoroscopy images using intensity-based MI, gradient-based MI and phase-based MI. Errors in the coordinates of fiducials present in the bone structures were used to assess the accuracy of the different registration schemes. Quantitative results demonstrate that the performance of intensity-based MI was the worst. Gradient-based MI performed slightly better, while phase-based MI results were the best consistently producing the lowest errors.
Bader, Brett William; Chew, Peter A.; Abdelali, Ahmed
2008-08-01
We describe an entirely statistics-based, unsupervised, and language-independent approach to multilingual information retrieval, which we call Latent Morpho-Semantic Analysis (LMSA). LMSA overcomes some of the shortcomings of related previous approaches such as Latent Semantic Analysis (LSA). LMSA has an important theoretical advantage over LSA: it combines well-known techniques in a novel way to break the terms of LSA down into units which correspond more closely to morphemes. Thus, it has a particular appeal for use with morphologically complex languages such as Arabic. We show through empirical results that the theoretical advantages of LMSA can translate into significant gains in precision in multilingual information retrieval tests. These gains are not matched either when a standard stemmer is used with LSA, or when terms are indiscriminately broken down into n-grams.
Meyer, Patrick E; Lafitte, Frédéric; Bontempi, Gianluca
2008-01-01
Results This paper presents the R/Bioconductor package minet (version 1.1.6) which provides a set of functions to infer mutual information networks from a dataset. Once fed with a microarray dataset, the package returns a network where nodes denote genes, edges model statistical dependencies between genes and the weight of an edge quantifies the statistical evidence of a specific (e.g transcriptional) gene-to-gene interaction. Four different entropy estimators are made available in the package minet (empirical, Miller-Madow, Schurmann-Grassberger and shrink) as well as four different inference methods, namely relevance networks, ARACNE, CLR and MRNET. Also, the package integrates accuracy assessment tools, like F-scores, PR-curves and ROC-curves in order to compare the inferred network with a reference one. Conclusion The package minet provides a series of tools for inferring transcriptional networks from microarray data. It is freely available from the Comprehensive R Archive Network (CRAN) as well as from the Bioconductor website. PMID:18959772
NASA Astrophysics Data System (ADS)
Liang, Yongfang; Chen, Hua-mei
2005-04-01
Joint histogram is the only quantity required to calculate the mutual information (MI) between two images. For MI based image registration, joint histograms are often estimated through linear interpolation or partial volume interpolation (PVI). It has been pointed out that both methods may result in a phenomenon known as interpolation induced artifacts. In this paper, we implemented a wide range of interpolation/approximation kernels for joint histogram estimation. Some kernels are nonnegative. In this case, these kernels are applied in two ways as the linear kernel is applied in linear interpolation and PVI. In addition, we implemented two other joint histogram estimation methods devised to overcome the interpolation artifact problem. They are nearest neighbor interpolation with jittered sampling with/without histogram blurring and data resampling. We used the clinical data obtained from Vanderbilt University for all of the experiments. The objective of this study is to perform a comprehensive comparison and evaluation of different joint histogram estimation methods for MI based image registration in terms of artifacts reduction and registration accuracy.
Power Spectrum and Mutual Information Analyses of DNA Base (Nucleotide) Sequences
NASA Astrophysics Data System (ADS)
Isohata, Yasuhiko; Hayashi, Masaki
2003-03-01
On the basis of the power spectrum analyses for the base (nucleotide) sequences of various genes, we have studied long-range correlations in total base sequences which are expressed as 1/fα, behaviour of the exponent α for the accumulated base sequences as well as periodicities at short range. In particular from the analysis of content rate distributions of α we have obtained the average value \\barα=0.40± 0.01 and \\barα=0.20± 0.01 for the human genes and S. cerevisiae genes, respectively. We have also performed the analyses using the mutual information function. We show that there exists a clear difference between the content rate distributions of correlation lengths for the sample human genes and the S. cerevisiae genes. We are led to a conjecture that the elongation of the correlation length in the base sequences of genes from the early eukaryote (S. cerevisiae) to the late eukaryote (human) should be the definite reflection of the evolutionary process.
NASA Astrophysics Data System (ADS)
Molina-Vilaplana, Javier; Sodano, Pasquale
2011-10-01
In ( d + 1) dimensional Multiscale Entanglement Renormalization Ansatz (MERA) networks, tensors are connected so as to reproduce the discrete, ( d + 2) holographic geometry of Anti de Sitter space (AdS d+2) with the original system lying at the boundary. We analyze the MERA renormalization flow that arises when computing the quantum correlations between two disjoint blocks of a quantum critical system, to show that the structure of the causal cones characteristic of MERA, requires a transition between two different regimes attainable by changing the ratio between the size and the separation of the two disjoint blocks. We argue that this transition in the MERA causal developments of the blocks may be easily accounted by an AdS d+2 black hole geometry when the mutual information is computed using the Ryu-Takayanagi formula. As an explicit example, we use a BTZ AdS3 black hole to compute the MI and the quantum correlations between two disjoint intervals of a one dimensional boundary critical system. Our results for this low dimensional system not only show the existence of a phase transition emerging when the conformal four point ratio reaches a critical value but also provide an intuitive entropic argument accounting for the source of this instability. We discuss the robustness of this transition when finite temperature and finite size effects are taken into account.
Estimation of Delta Wave by Mutual Information of Heartbeat During Sleep
NASA Astrophysics Data System (ADS)
Kurihara, Yosuke; Watanabe, Kajiro; Kobayashi, Kazuyuki; Tanaka, Hiroshi
The quality of sleep is evaluated based on the sleep stages judged by R-K method or the manual of American Academy of Sleep Medicine. The brainwaves, eye movements, and chin EMG of sleeping subjects are used for the judgment. These methods above, however, require some electrodes to be attached to the head and the face to obtain the brainwaves, eye movements, and chin EMG, thus making the measurements troublesome to be held on a daily basis. If non-invasive measurements of brainwaves, eye movements, and chin EMG are feasible, or their equivalent data can be estimated through other bio-signals, the monitoring of the quality of daily sleeps, which influences the health condition, will be easy. In this paper, we discuss the appearance rate of delta wave occurrences, which is deeply related with the depth of sleep, can be estimated based on the average amount of mutual information calculated by pulse wave signals and body movements measured non-invasively by the pneumatic method. As a result, the root mean square error between the appearance rate of delta wave occurrences measured with a polysomnography and the estimated delta pulse was 14.93%.
Zhou, G D
2006-06-01
In this paper, we present a biomedical name recognition system, called PowerBioNE. In order to deal with the special phenomena in the biomedical domain, various evidential features are proposed and integrated through a mutual information independence model (MIIM). In addition, a support vector machine (SVM) plus sigmoid is proposed to resolve the data sparseness problem in the MIIM. In this way, the data sparseness problem in MIIM-based biomedical name recognition can be resolved effectively and a biomedical name recognition system with better performance and better portability can be achieved. Finally, we present two post-processing modules to deal with the nested entity name and abbreviation phenomena in the biomedical domain to further improve the performance. Evaluation shows that our system achieves F-measures of 69.1 and 71.2 on the 23 classes of GENIA V1.1 and V3.0, respectively. In particular, our system achieves an F-measure of 77.8 on the "protein" class of GENIA V3.0. It also shows that our system outperforms the best-reported system on GENIA V1.1 and V3.0. PMID:16112894
Monotonicity of the unified quantum ( r, s)-entropy and ( r, s)-mutual information
NASA Astrophysics Data System (ADS)
Fan, Ya-Jing; Cao, Huai-Xin
2015-12-01
Monotonicity of the unified quantum ( r, s)-entropy Ers(ρ ) and the unified quantum ( r, s)-mutual information Irs(ρ ) is discussed in this paper. Some basic properties of them are explored, and the following conclusions are established. (1) For any 0
Abásolo, D; Escudero, J; Hornero, R; Gómez, C; Espino, P
2008-10-01
We analysed the electroencephalogram (EEG) from Alzheimer's disease (AD) patients with two nonlinear methods: approximate entropy (ApEn) and auto mutual information (AMI). ApEn quantifies regularity in data, while AMI detects linear and nonlinear dependencies in time series. EEGs from 11 AD patients and 11 age-matched controls were analysed. ApEn was significantly lower in AD patients at electrodes O1, O2, P3 and P4 (p < 0.01). The EEG AMI decreased more slowly with time delays in patients than in controls, with significant differences at electrodes T5, T6, O1, O2, P3 and P4 (p < 0.01). The strong correlation between results from both methods shows that the AMI rate of decrease can be used to estimate the regularity in time series. Our work suggests that nonlinear EEG analysis may contribute to increase the insight into brain dysfunction in AD, especially when different time scales are inspected, as is the case with AMI. PMID:18784948
Geometrical mutual information at the tricritical point of the two-dimensional Blume–Capel model
NASA Astrophysics Data System (ADS)
Mandal, Ipsita; Inglis, Stephen; Melko, Roger G.
2016-07-01
The spin-1 classical Blume–Capel model on a square lattice is known to exhibit a finite-temperature phase transition described by the tricritical Ising CFT in 1 + 1 space-time dimensions. This phase transition can be accessed with classical Monte Carlo simulations, which, via a replica-trick calculation, can be used to study the shape-dependence of the classical Rényi entropies for a torus divided into two cylinders. From the second Rényi entropy, we calculate the geometrical mutual information (GMI) introduced by Stéphan et al (2014 Phys. Rev. Lett. 112 127204) and use it to extract a numerical estimate for the value of the central charge near the tricritical point. By comparing to the known CFT result, c = 7/10, we demonstrate how this type of GMI calculation can be used to estimate the position of the tricritical point in the phase diagram.
Holographic mutual information and distinguishability of Wilson loop and defect operators
NASA Astrophysics Data System (ADS)
Hartnoll, Sean A.; Mahajan, Raghu
2015-02-01
The mutual information of disconnected regions in large N gauge theories with holographic gravity duals can undergo phase transitions. These occur when connected and disconnected bulk Ryu-Takayanagi surfaces exchange dominance. That is, the bulk `soap bubble' snaps as the boundary regions are drawn apart. We give a gauge-theoretic characterization of this transition: States with and without a certain defect operator insertion — the defect separates the entangled spatial regions — are shown to be perfectly distinguishable if and only if the Ryu-Takayanagi surface is connected. Meanwhile, states with and without a certain Wilson loop insertion — the Wilson loop nontrivially threads the spatial regions — are perfectly distinguishable if and only if the Ryu-Takayanagi surface is disconnected. The quantum relative entropy of two perfectly distinguishable states is infinite. The results are obtained by relating the soap bubble transition to Hawking-Page (deconfinement) transitions in the Rényi entropies, where defect operators and Wilson loops are known to act as order parameters.
Genomic signatures in viral sequences by in-frame and out-frame mutual information.
Serrano-Solís, Víctor; Cocho, Germinal; José, Marco V
2016-08-21
In order to understand the unique biology of viruses, we use the Mutual Information Function (MIF) to characterize 792 viral sequences comprising 458 viral whole genomes. A 3-base periodicity (3-bp) was observed only in DNA-viruses whereas RNA-viruses showed irregular patterns. The correlation of MIF values at frequencies of 3-bp (in-frame) with frequencies of 4 and 5bps (out-frame), turned out to be useful to distinguish viruses according to their respective taxonomic order, and whether they pertain to any of the three different kingdoms, Eubacteria, Archaea and Eukarya. The clustering of viruses was carried out by the use of a new statistics, namely, the pair of in- and out-frame values of the MIF. The clustering thus obtained turned out to be entirely consistent with the current viral taxonomy. As a result we were able to compare in a single plot both viral and cellular genomes unlike any given phylogenetic reconstruction. PMID:27178876
NASA Astrophysics Data System (ADS)
Hamrouni, Sameh; Rougon, Nicolas; Pr"teux, Françoise
2011-03-01
In perfusion MRI (p-MRI) exams, short-axis (SA) image sequences are captured at multiple slice levels along the long-axis of the heart during the transit of a vascular contrast agent (Gd-DTPA) through the cardiac chambers and muscle. Compensating cardio-thoracic motions is a requirement for enabling computer-aided quantitative assessment of myocardial ischaemia from contrast-enhanced p-MRI sequences. The classical paradigm consists of registering each sequence frame on a reference image using some intensity-based matching criterion. In this paper, we introduce a novel unsupervised method for the spatio-temporal groupwise registration of cardiac p-MRI exams based on normalized mutual information (NMI) between high-dimensional feature distributions. Here, local contrast enhancement curves are used as a dense set of spatio-temporal features, and statistically matched through variational optimization to a target feature distribution derived from a registered reference template. The hard issue of probability density estimation in high-dimensional state spaces is bypassed by using consistent geometric entropy estimators, allowing NMI to be computed directly from feature samples. Specifically, a computationally efficient kth-nearest neighbor (kNN) estimation framework is retained, leading to closed-form expressions for the gradient flow of NMI over finite- and infinite-dimensional motion spaces. This approach is applied to the groupwise alignment of cardiac p-MRI exams using a free-form Deformation (FFD) model for cardio-thoracic motions. Experiments on simulated and natural datasets suggest its accuracy and robustness for registering p-MRI exams comprising more than 30 frames.
Mutual information analysis of sleep EEG in detecting psycho-physiological insomnia.
Aydın, Serap; Tunga, M Alper; Yetkin, Sinan
2015-05-01
The primary goal of this study is to state the clear changes in functional brain connectivity during all night sleep in psycho-physiological insomnia (PPI). The secondary goal is to investigate the usefulness of Mutual Information (MI) analysis in estimating cortical sleep EEG arousals for detection of PPI. For these purposes, healthy controls and patients were compared to each other with respect to both linear (Pearson correlation coefficient and coherence) and nonlinear quantifiers (MI) in addition to phase locking quantification for six sleep stages (stage.1-4, rem, wake) by means of interhemispheric dependency between two central sleep EEG derivations. In test, each connectivity estimation calculated for each couple of epoches (C3-A2 and C4-A1) was identified by the vector norm of estimation. Then, patients and controls were classified by using 10 different types of data mining classifiers for five error criteria such as accuracy, root mean squared error, sensitivity, specificity and precision. High performance in a classification through a measure will validate high contribution of that measure to detecting PPI. The MI was found to be the best method in detecting PPI. In particular, the patients had lower MI, higher PCC for all sleep stages. In other words, the lower sleep EEG synchronization suffering from PPI was observed. These results probably stand for the loss of neurons that then contribute to less complex dynamical processing within the neural networks in sleep disorders an the functional central brain connectivity is nonlinear during night sleep. In conclusion, the level of cortical hemispheric connectivity is strongly associated with sleep disorder. Thus, cortical communication quantified in all existence sleep stages might be a potential marker for sleep disorder induced by PPI. PMID:25732074
Algorithm and program for information processing with the filin apparatus
NASA Technical Reports Server (NTRS)
Gurin, L. S.; Morkrov, V. S.; Moskalenko, Y. I.; Tsoy, K. A.
1979-01-01
The reduction of spectral radiation data from space sources is described. The algorithm and program for identifying segments of information obtained from the Film telescope-spectrometer on the Salyut-4 are presented. The information segments represent suspected X-ray sources. The proposed algorithm is an algorithm of the lowest level. Following evaluation, information free of uninformative segments is subject to further processing with algorithms of a higher level. The language used is FORTRAN 4.
An Innovative Thinking-Based Intelligent Information Fusion Algorithm
Hu, Liang; Liu, Gang; Zhou, Jin
2013-01-01
This study proposes an intelligent algorithm that can realize information fusion in reference to the relative research achievements in brain cognitive theory and innovative computation. This algorithm treats knowledge as core and information fusion as a knowledge-based innovative thinking process. Furthermore, the five key parts of this algorithm including information sense and perception, memory storage, divergent thinking, convergent thinking, and evaluation system are simulated and modeled. This algorithm fully develops innovative thinking skills of knowledge in information fusion and is a try to converse the abstract conception of brain cognitive science to specific and operable research routes and strategies. Furthermore, the influences of each parameter of this algorithm on algorithm performance are analyzed and compared with those of classical intelligent algorithms trough test. Test results suggest that the algorithm proposed in this study can obtain the optimum problem solution by less target evaluation times, improve optimization effectiveness, and achieve the effective fusion of information. PMID:23956699
Information Theory, Inference and Learning Algorithms
NASA Astrophysics Data System (ADS)
Mackay, David J. C.
2003-10-01
Information theory and inference, often taught separately, are here united in one entertaining textbook. These topics lie at the heart of many exciting areas of contemporary science and engineering - communication, signal processing, data mining, machine learning, pattern recognition, computational neuroscience, bioinformatics, and cryptography. This textbook introduces theory in tandem with applications. Information theory is taught alongside practical communication systems, such as arithmetic coding for data compression and sparse-graph codes for error-correction. A toolbox of inference techniques, including message-passing algorithms, Monte Carlo methods, and variational approximations, are developed alongside applications of these tools to clustering, convolutional codes, independent component analysis, and neural networks. The final part of the book describes the state of the art in error-correcting codes, including low-density parity-check codes, turbo codes, and digital fountain codes -- the twenty-first century standards for satellite communications, disk drives, and data broadcast. Richly illustrated, filled with worked examples and over 400 exercises, some with detailed solutions, David MacKay's groundbreaking book is ideal for self-learning and for undergraduate or graduate courses. Interludes on crosswords, evolution, and sex provide entertainment along the way. In sum, this is a textbook on information, communication, and coding for a new generation of students, and an unparalleled entry point into these subjects for professionals in areas as diverse as computational biology, financial engineering, and machine learning.
Sun, Leiming; Wang, Chan
2016-01-01
Background. Genome-wide association studies have succeeded in detecting novel common variants which associate with complex diseases. As a result of the fast changes in next generation sequencing technology, a large number of sequencing data are generated, which offers great opportunities to identify rare variants that could explain a larger proportion of missing heritability. Many effective and powerful methods are proposed, although they are usually limited to continuous, dichotomous or ordinal traits. Notice that traits having nominal categorical features are commonly observed in complex diseases, especially in mental disorders, which motivates the incorporation of the characteristics of the categorical trait into association studies with rare and common variants. Methods. We construct two simple and intuitive nonparametric tests, MIT and aMIT, based on mutual information for detecting association between genetic variants in a gene or region and a categorical trait. MIT and aMIT can gauge the difference among the distributions of rare and common variants across a region given every categorical trait value. If there is little association between variants and a categorical trait, MIT or aMIT approximately equals zero. The larger the difference in distributions, the greater values MIT and aMIT have. Therefore, MIT and aMIT have the potential for detecting functional variants. Results.We checked the validity of proposed statistics and compared them to the existing ones through extensive simulation studies with varied combinations of the numbers of variants of rare causal, rare non-causal, common causal, and common non-causal, deleterious and protective, various minor allele frequencies and different levels of linkage disequilibrium. The results show our methods have higher statistical power than conventional ones, including the likelihood based score test, in most cases: (1) there are multiple genetic variants in a gene or region; (2) both protective and deleterious
Exploring a new best information algorithm for Iliad.
Guo, D.; Lincoln, M. J.; Haug, P. J.; Turner, C. W.; Warner, H. R.
1991-01-01
Iliad is a diagnostic expert system for internal medicine. One important feature that Iliad offers is the ability to analyze a particular patient case and to determine the most cost-effective method for pursuing the work-up. Iliad's current "best information" algorithm has not been previously validated and compared to other potential algorithms. Therefore, this paper presents a comparison of four new algorithms to the current algorithm. The basis for this comparison was eighteen "vignette" cases derived from real patient cases from the University of Utah Medical Center. The results indicated that the current algorithm can be significantly improved. More promising algorithms are suggested for future investigation. PMID:1807677
Jani, S; Kishan, A; O'Connell, D; King, C; Steinberg, M; Low, D; Lamb, J
2014-06-01
Purpose: To investigate if pelvic nodal coverage for prostate patients undergoing intensity modulated radiotherapy (IMRT) can be predicted using mutual image information computed between planning and cone-beam CTs (CBCTs). Methods: Four patients with high-risk prostate adenocarcinoma were treated with IMRT on a Varian TrueBeam. Plans were designed such that 95% of the nodal planning target volume (PTV) received the prescription dose of 45 Gy (N=1) or 50.4 Gy (N=3). Weekly CBCTs (N=25) were acquired and the nodal clinical target volumes and organs at risk were contoured by a physician. The percent nodal volume receiving prescription dose was recorded as a ground truth. Using the recorded shifts performed by the radiation therapists at the time of image acquisition, CBCTs were aligned with the planning kVCT. Mutual image information (MI) was calculated between the CBCT and the aligned planning CT within the contour of the nodal PTV. Due to variable CBCT fields-of-view, CBCT images covering less than 90% of the nodal volume were excluded from the analysis, resulting in the removal of eight CBCTs. Results: A correlation coefficient of 0.40 was observed between the MI metric and the percent of the nodal target volume receiving the prescription dose. One patient's CBCTs had clear outliers from the rest of the patients. Upon further investigation, we discovered image artifacts that were present only in that patient's images. When those four images were excluded, the correlation improved to 0.81. Conclusion: This pilot study shows the potential of predicting pelvic nodal dosimetry by computing the mutual image information between planning CTs and patient setup CBCTs. Importantly, this technique does not involve manual or automatic contouring of the CBCT images. Additional patients and more robust exclusion criteria will help validate our findings.
Information filtering via weighted heat conduction algorithm
NASA Astrophysics Data System (ADS)
Liu, Jian-Guo; Guo, Qiang; Zhang, Yi-Cheng
2011-06-01
In this paper, by taking into account effects of the user and object correlations on a heat conduction (HC) algorithm, a weighted heat conduction (WHC) algorithm is presented. We argue that the edge weight of the user-object bipartite network should be embedded into the HC algorithm to measure the object similarity. The numerical results indicate that both the accuracy and diversity could be improved greatly compared with the standard HC algorithm and the optimal values reached simultaneously. On the Movielens and Netflix datasets, the algorithmic accuracy, measured by the average ranking score, can be improved by 39.7% and 56.1% in the optimal case, respectively, and the diversity could reach 0.9587 and 0.9317 when the recommendation list equals to 5. Further statistical analysis indicates that, in the optimal case, the distributions of the edge weight are changed to the Poisson form, which may be the reason why HC algorithm performance could be improved. This work highlights the effect of edge weight on a personalized recommendation study, which maybe an important factor affecting personalized recommendation performance.
NASA Astrophysics Data System (ADS)
Kurihara, Yosuke; Watanabe, Kajiro; Kobayashi, Kazuyuki; Tanaka, Tanaka
Sleep disorders disturb the recovery from mental and physical fatigues, one of the functions of the sleep. The majority of those who with the disorders are suffering from Sleep Apnea Syndrome (SAS). Continuous Hypoxia during sleep due to SAS cause Circulatory Disturbances, such as hypertension and ischemic heart disease, and Malfunction of Autonomic Nervous System, and other severe complications, often times bringing the suffers to death. In order to prevent these from happening, it is important to detect the SAS in its early stage by monitoring the daily respirations during sleep, and to provide appropriate treatments at medical institutions. In this paper, the Pneumatic Method to detect the Apnea period during sleep is proposed. Pneumatic method can measure heartbeat and respiration signal. Respiration signal can be considered as noise against heartbeat signal, and the decrease in the respiration signal due to Apnea increases the Average Mutual Information of heartbeat. The result of scaling analysis of the average mutual information is defined as threshold to detect the apnea period. The root mean square error between the lengths of Apnea measured by Strain Gauge using for reference and those measured by using the proposed method was 3.1 seconds. And, error of the number of apnea times judged by doctor and proposal method in OSAS patients was 3.3 times.
Misra, Sanchit; Pamnany, Kiran; Aluru, Srinivas
2015-01-01
Construction of whole-genome networks from large-scale gene expression data is an important problem in systems biology. While several techniques have been developed, most cannot handle network reconstruction at the whole-genome scale, and the few that can, require large clusters. In this paper, we present a solution on the Intel Xeon Phi coprocessor, taking advantage of its multi-level parallelism including many x86-based cores, multiple threads per core, and vector processing units. We also present a solution on the Intel® Xeon® processor. Our solution is based on TINGe, a fast parallel network reconstruction technique that uses mutual information and permutation testing for assessing statistical significance. We demonstrate the first ever inference of a plant whole genome regulatory network on a single chip by constructing a 15,575 gene network of the plant Arabidopsis thaliana from 3,137 microarray experiments in only 22 minutes. In addition, our optimization for parallelizing mutual information computation on the Intel Xeon Phi coprocessor holds out lessons that are applicable to other domains. PMID:26451815
NASA Astrophysics Data System (ADS)
Zhou, Tianci; Chen, Xiao; Fradkin, Eduardo
We investigate the entanglement entropy(EE) of circular entangling surfaces in the 2+1d quantum Lifshitz model, where the spatially conformal invariant ground state is a Rokhsar-Kivelson state with Gibbs weight of 2d free Boson. We use cut-off independent mutual information regulator to define and calculate the subleading correction in the EE. The subtlety due to the Boson compactification in the replica trick is carefully taken care of. Our results show that for circular entangling surface, the subleading term is a constant on both the sphere of arbitrary radius and infinite plane. For the latter case, it parallels the constancy of disk EE in 2+1d conformal field theory, despite the lack of full space time conformal invariance. In the end, we present the mutual information of two disjoint disks and compare its scaling function in the small parameter regime (radii much smaller than their separation) with Cardy's general CFT results. This work was supported in part by the National Science Foundation Grants NSF-DMR-13-06011(TZ) and DMR-1408713 (XC, EF).
ERIC Educational Resources Information Center
Wang, Chun
2013-01-01
Cognitive diagnostic computerized adaptive testing (CD-CAT) purports to combine the strengths of both CAT and cognitive diagnosis. Cognitive diagnosis models aim at classifying examinees into the correct mastery profile group so as to pinpoint the strengths and weakness of each examinee whereas CAT algorithms choose items to determine those…
Fast method of homology and purine-pyrimidine mutual relations between DNA sequences search.
Korotkov, E V
1994-01-01
A new algorithm for scanning sequences is described. This algorithm uses the boolean operators AND and OR. The mutual information between the sequences is used as a measure of sequence interrelation. It allows evaluation of the probability of accidental sequence interrelation in a quantitative manner. The proposed algorithm was used for searching for MB1 repeats in human and other mammalian sequences. PMID:7841466
Chen, Chao; Yan, Xuefeng
2015-06-01
In this paper, an optimized multilayer feed-forward network (MLFN) is developed to construct a soft sensor for controlling naphtha dry point. To overcome the two main flaws in the structure and weight of MLFNs, which are trained by a back-propagation learning algorithm, minimal redundancy maximal relevance-partial mutual information clustering (mPMIc) integrated with least square regression (LSR) is proposed to optimize the MLFN. The mPMIc can determine the location of hidden layer nodes using information in the hidden and output layers, as well as remove redundant hidden layer nodes. These selected nodes are highly related to output data, but are minimally correlated with other hidden layer nodes. The weights between the selected hidden layer nodes and output layer are then updated through LSR. When the redundant nodes from the hidden layer are removed, the ideal MLFN structure can be obtained according to the test error results. In actual applications, the naphtha dry point must be controlled accurately because it strongly affects the production yield and the stability of subsequent operational processes. The mPMIc-LSR MLFN with a simple network size performs better than other improved MLFN variants and existing efficient models. PMID:25055386
Information content of ozone retrieval algorithms
NASA Technical Reports Server (NTRS)
Rodgers, C.; Bhartia, P. K.; Chu, W. P.; Curran, R.; Deluisi, J.; Gille, J. C.; Hudson, R.; Mateer, C.; Rusch, D.; Thomas, R. J.
1989-01-01
The algorithms are characterized that were used for production processing by the major suppliers of ozone data to show quantitatively: how the retrieved profile is related to the actual profile (This characterizes the altitude range and vertical resolution of the data); the nature of systematic errors in the retrieved profiles, including their vertical structure and relation to uncertain instrumental parameters; how trends in the real ozone are reflected in trends in the retrieved ozone profile; and how trends in other quantities (both instrumental and atmospheric) might appear as trends in the ozone profile. No serious deficiencies were found in the algorithms used in generating the major available ozone data sets. As the measurements are all indirect in someway, and the retrieved profiles have different characteristics, data from different instruments are not directly comparable.
Guinea-Martin, Daniel; Mora, Ricardo; Ruiz-Castillo, Javier
2015-01-01
In this article, we study the effects of ethnicity and gender on occupational segregation. Traditionally, researchers have examined the two sources of segregation separately. In contrast, we measure their joint effect by applying a multigroup segregation index-the Mutual Information or M index-to the product of the seven ethnic groups and two genders distinguished in our 2001 Census data for England and Wales. We exploit M's additive decomposability property to pose the following two questions: (i) Is there an interaction effect? (ii) How much does each source contribute to occupational segregation, controlling for the effect of the other? Although the role of ethnicity is non-negligible in the areas where minorities are concentrated, our findings confirm the greater importance of gender over ethnicity as a source of segregation. Moreover, we find a small "dwindling" interaction effect between the two sources of segregation: ethnicity slightly weakens the segregating power of gender and vice versa. PMID:25432611
Yang, Bin; Zhang, Wei; Wang, Haifeng; Song, Chuandong; Chen, Yuehui
2016-05-01
Regulatory interactions among target genes and regulatory factors occur instantaneously or with time-delay. In this paper, we propose a novel approach namely TDSDMI based on time-delayed S-system model (TDSS) model and delayed mutual information (DMI) to infer time-delay gene regulatory network (TDGRN). Firstly DMI is proposed to delete redundant regulator factors for each target gene. Secondly restricted gene expression programming (RGEP) is proposed as a new representation of the TDSS model to identify instantaneous and time-delayed interactions. To verify the effectiveness of the proposed method, TDSDMI is applied to both simulated and real biological datasets. Experimental results reveal that TDSDMI performs better than the recent reconstruction methods. PMID:27058285
NASA Astrophysics Data System (ADS)
Wang, Xixi; Nagarajan, Mahesh B.; Abidin, Anas Z.; DSouza, Adora; Hobbs, Susan K.; Wismüller, Axel
2015-03-01
Functional MRI (fMRI) is currently used to investigate structural and functional connectivity in human brain networks. To this end, previous studies have proposed computational methods that involve assumptions that can induce information loss, such as assumed linear coupling of the fMRI signals or requiring dimension reduction. This study presents a new computational framework for investigating the functional connectivity in the brain and recovering network structure while reducing the information loss inherent in previous methods. For this purpose, pair-wise mutual information (MI) was extracted from all pixel time series within the brain on resting-state fMRI data. Non-metric topographic mapping of proximity (TMP) data was subsequently applied to recover network structure from the pair-wise MI analysis. Our computational framework is demonstrated in the task of identifying regions of the primary motor cortex network on resting state fMRI data. For ground truth comparison, we also localized regions of the primary motor cortex associated with hand movement in a task-based fMRI sequence with a finger-tapping stimulus function. The similarity between our pair-wise MI clustering results and the ground truth is evaluated using the dice coefficient. Our results show that non-metric clustering with the TMP algorithm, as performed on pair-wise MI analysis, was able to detect the primary motor cortex network and achieved a dice coefficient of 0.53 in terms of overlap with the ground truth. Thus, we conclude that our computational framework can extract and visualize valuable information concerning the underlying network structure between different regions of the brain in resting state fMRI.
Fuzzy Information Retrieval Using Genetic Algorithms and Relevance Feedback.
ERIC Educational Resources Information Center
Petry, Frederick E.; And Others
1993-01-01
Describes an approach that combines concepts from information retrieval, fuzzy set theory, and genetic programing to improve weighted Boolean query formulation via relevance feedback. Highlights include background on information retrieval systems; genetic algorithms; subproblem formulation; and preliminary results based on a testbed. (Contains 12…
Use of mutual information to decrease entropy: Implications for the second law of thermodynamics
Lloyd, S.
1989-05-15
Several theorems on the mechanics of gathering information are proved, and the possibility of violating the second law of thermodynamics by obtaining information is discussed in light of these theorems. Maxwell's demon can lower the entropy of his surroundings by an amount equal to the difference between the maximum entropy of his recording device and its initial entropy, without generating a compensating entropy increase. A demon with human-scale recording devices can reduce the entropy of a gas by a negligible amount only, but the proof of the demon's impracticability leaves open the possibility that systems highly correlated with their environment can reduce the environment's entropy by a substantial amount without increasing entropy elsewhere. In the event that a boundary condition for the universe requires it to be in a state of low entropy when small, the correlations induced between different particle modes during the expansion phase allow the modes to behave like Maxwell's demons during the contracting phase, reducing the entropy of the universe to a low value.
Wang, Luman; Mo, Qiaochu; Wang, Jianxin
2015-01-01
Most current gene coexpression databases support the analysis for linear correlation of gene pairs, but not nonlinear correlation of them, which hinders precisely evaluating the gene-gene coexpression strengths. Here, we report a new database, MIrExpress, which takes advantage of the information theory, as well as the Pearson linear correlation method, to measure the linear correlation, nonlinear correlation, and their hybrid of cell-specific gene coexpressions in immune cells. For a given gene pair or probe set pair input by web users, both mutual information (MI) and Pearson correlation coefficient (r) are calculated, and several corresponding values are reported to reflect their coexpression correlation nature, including MI and r values, their respective rank orderings, their rank comparison, and their hybrid correlation value. Furthermore, for a given gene, the top 10 most relevant genes to it are displayed with the MI, r, or their hybrid perspective, respectively. Currently, the database totally includes 16 human cell groups, involving 20,283 human genes. The expression data and the calculated correlation results from the database are interactively accessible on the web page and can be implemented for other related applications and researches. PMID:26881263
Wang, Luman; Mo, Qiaochu; Wang, Jianxin
2015-01-01
Most current gene coexpression databases support the analysis for linear correlation of gene pairs, but not nonlinear correlation of them, which hinders precisely evaluating the gene-gene coexpression strengths. Here, we report a new database, MIrExpress, which takes advantage of the information theory, as well as the Pearson linear correlation method, to measure the linear correlation, nonlinear correlation, and their hybrid of cell-specific gene coexpressions in immune cells. For a given gene pair or probe set pair input by web users, both mutual information (MI) and Pearson correlation coefficient (r) are calculated, and several corresponding values are reported to reflect their coexpression correlation nature, including MI and r values, their respective rank orderings, their rank comparison, and their hybrid correlation value. Furthermore, for a given gene, the top 10 most relevant genes to it are displayed with the MI, r, or their hybrid perspective, respectively. Currently, the database totally includes 16 human cell groups, involving 20,283 human genes. The expression data and the calculated correlation results from the database are interactively accessible on the web page and can be implemented for other related applications and researches. PMID:26881263
Mutual information reveals multiple structural relaxation mechanisms in a model glass former
Dunleavy, Andrew J.; Wiesner, Karoline; Yamamoto, Ryoichi; Royall, C. Patrick
2015-01-01
Among the key challenges to our understanding of solidification in the glass transition is that it is accompanied by little apparent change in structure. Recently, geometric motifs have been identified in glassy liquids, but a causal link between these motifs and solidification remains elusive. One ‘smoking gun’ for such a link would be identical scaling of structural and dynamic lengthscales on approaching the glass transition, but this is highly controversial. Here we introduce an information theoretic approach to determine correlations in displacement for particle relaxation encoded in the initial configuration of a glass-forming liquid. We uncover two populations of particles, one inclined to relax quickly, the other slowly. Each population is correlated with local density and geometric motifs. Our analysis further reveals a dynamic lengthscale similar to that associated with structural properties, which may resolve the discrepancy between structural and dynamic lengthscales. PMID:25608791
MINT: Mutual Information Based Transductive Feature Selection for Genetic Trait Prediction.
He, Dan; Rish, Irina; Haws, David; Parida, Laxmi
2016-01-01
Whole genome prediction of complex phenotypic traits using high-density genotyping arrays has attracted a lot of attention, as it is relevant to the fields of plant and animal breeding and genetic epidemiology. Since the number of genotypes is generally much bigger than the number of samples, predictive models suffer from the curse of dimensionality. The curse of dimensionality problem not only affects the computational efficiency of a particular genomic selection method, but can also lead to a poor performance, mainly due to possible overfitting, or un-informative features. In this work, we propose a novel transductive feature selection method, called MINT, which is based on the MRMR (Max-Relevance and Min-Redundancy) criterion. We apply MINT on genetic trait prediction problems and show that, in general, MINT is a better feature selection method than the state-of-the-art inductive method MRMR. PMID:27295642
Inversion of Magnetotelluric Data in Anisotropic Media Using Maximization of Mutual Information
NASA Astrophysics Data System (ADS)
Mandolesi, E.; Jones, A. G.
2011-12-01
Regularization in inverse geophysics problems has been used extensively, due to the necessity to constrain the model space and to reduce the ill-posedness of several problems. Magnetotelluric (MT) problems suffer from severe non-linearity and ill-posedness, which makes MT inversions extremely challenging. The use of a reference model has been used by many authors in order to drive the inversion process to converge on a model that shares features with the reference, as a result reducing non-uniqueness and improving the model resolution. In our work the reference model drives the inversion keeping the conductivity distribution close to that of the velocity using variation of information as measure of distance between the two pictures. In this way the electrical conductivity and seismic velocity can be compared from a statistical point of view, without the necessity of a common parameterization or a strict geometrical similarity. Our work involves the inversion of MT long-period data, which are sensitive to electrical conductivity, using shear wave velocity maps as reference model in a 1D anisotropic domain. Computation of variation of information is performed through the generation of the joint probability distribution, which allows exploration of the relation between models that fit seismic data and models that fit electrical properties. An approximate agreement between geoelectric strike direction and seismic fast axis have been recognized in different continental lithospheric areas, suggesting a common cause for both the seismic and electric anisotropic behavior. We present an application of this inversion approach to a real dataset from Central Germany, discussing pros and cons of this approach in relation to similar studies on the same area. Due to the minimal assumptions required by this approach, it highlights the possibility of application to different tomography techniques.
Feng, Yanqiu; Chen, Wufan
2005-01-01
PROPELLER (Periodically Rotated Overlapping ParallEl Lines with Enhanced Reconstruction) MRI, proposed by J. G. Pipe [1], offers a novel and effective means for compensating motion. For the reconstruction of PROPLLER data, algorithms to reliably and accurately extract inter-strip motion from data in central overlapped area are crucial to motion artifacts suppression. When implemented on T1-weighted MR data, the reconstruction algorithm, with motion estimated by registration based on maximizing correlation energy in frequency domain (CF), produces images with low quality due to the inaccurate estimation of motion. In this paper, a new algorithm is proposed for motion estimation based on the registration by maximizing mutual information in spatial domain (MIS). Furthermore, the optimization process is initialized by CF algorithm, so the algorithm is abbreviated as CF-MIS algorithm in this paper. With phantom and in vivo MR imaging, the CF-MIS algorithm was shown to be of higher accuracy in rotation estimation than CF algorithm. Consequently, the head motion in T1-weighted PROPELLER MRI was better corrected. PMID:17282454
NASA Astrophysics Data System (ADS)
Ji, Songbai; Hartov, Alex; Roberts, David; Paulsen, Keith
2008-03-01
Intraoperative ultrasound (iUS) has emerged as a practical neuronavigational tool for brain shift compensation in image-guided tumor resection surgeries. The use of iUS is optimized when coregistered with preoperative magnetic resonance images (pMR) of the patient's head. However, the fiducial-based registration alone does not necessarily optimize the alignment of internal anatomical structures deep in the brain (e.g., tumor) between iUS and pMR. In this paper, we investigated and evaluated an image-based re-registration scheme to maximize the normalized mutual information (nMI) between iUS and pMR to improve tumor boundary alignment using the fiducial registration as a starting point for optimization. We show that this scheme significantly (p<<0.001) reduces tumor boundary misalignment pre-durotomy. The same technique was employed to measure tumor displacement post-durotomy, and the locally measured tumor displacement was assimilated into a biomechanical model to estimate whole-brain deformation. Our results demonstrate that the nMI re-registration pre-durotomy is critical for obtaining accurate measurement of tumor displacement, which significantly improved model response at the craniotomy when compared with stereopsis data acquired independently from the tumor registration. This automatic and computationally efficient (<2min) re-registration technique is feasible for routine clinical use in the operating room (OR).
NASA Astrophysics Data System (ADS)
Kawamura, Tetsuo; Horio, Yoshihiko; Hasegawa, Mikio
The tabu search was implemented on a neural network with chaotic neuro-dynamics. This chaotic exponential tabu search shows great performance in solving quadratic assignment problems (QAPs). To exploit inherent parallel processing abilities of analog hardware systems, a synchronous updating scheme, where all the neurons in the network are updated at the same time, was proposed. However, several neurons may fire simultaneously with the synchronous updating. As a result, we cannot determine only one candidate for the 2-opt exchange from the many fired neurons. To solve this problem, several neuron selection methods, which select one specific neuron among the fired neurons, were proposed. These neuron selection methods improved the performance of the synchronous updating scheme. In this paper, we analyze the dynamics of the chaotic neural network with the neuron selection methods by means of the spatial and temporal mutual information. Through the analyses, the network solution search dynamics of the exponential chaotic tabu search with different neuron selection methods are evaluated.
Amin, Ruhul; Islam, S K Hafizul; Biswas, G P; Khan, Muhammad Khurram; Obaidat, Mohammad S
2015-11-01
In order to access remote medical server, generally the patients utilize smart card to login to the server. It has been observed that most of the user (patient) authentication protocols suffer from smart card stolen attack that means the attacker can mount several common attacks after extracting smart card information. Recently, Lu et al.'s proposes a session key agreement protocol between the patient and remote medical server and claims that the same protocol is secure against relevant security attacks. However, this paper presents several security attacks on Lu et al.'s protocol such as identity trace attack, new smart card issue attack, patient impersonation attack and medical server impersonation attack. In order to fix the mentioned security pitfalls including smart card stolen attack, this paper proposes an efficient remote mutual authentication protocol using smart card. We have then simulated the proposed protocol using widely-accepted AVISPA simulation tool whose results make certain that the same protocol is secure against active and passive attacks including replay and man-in-the-middle attacks. Moreover, the rigorous security analysis proves that the proposed protocol provides strong security protection on the relevant security attacks including smart card stolen attack. We compare the proposed scheme with several related schemes in terms of computation cost and communication cost as well as security functionalities. It has been observed that the proposed scheme is comparatively better than related existing schemes. PMID:26324169
Sosa, Daniela; Miramontes, Pedro; Li, Wentian; Mireles, Víctor; Bobadilla, Juan R.; José, Marco V.
2013-01-01
Recently, Trifonov's group proposed a 10-mer DNA motif YYYYYRRRRR as a solution of the long-standing problem of sequence-based nucleosome positioning. To test whether this generic decamer represents a biological meaningful signal, we compare the distribution of this motif in primates and Archaea, which are known to contain nucleosomes, and in Eubacteria, which do not possess nucleosomes. The distribution of the motif is analyzed by the mutual information function (MIF) with a shifted version of itself (MIF profile). We found common features in the patterns of this generic decamer on MIF profiles among primate species, and interestingly we found conspicuous but dissimilar MIF profiles for each Archaea tested. The overall MIF profiles for each chromosome in each primate species also follow a similar pattern. Trifonov's generic decamer may be a highly conserved motif for the nucleosome positioning, but we argue that this is not the only motif. The distribution of this generic decamer exhibits previously unidentified periodicities, which are associated to highly repetitive sequences in the genome. Alu repetitive elements contribute to the most fundamental structure of nucleosome positioning in higher Eukaryotes. In some regions of primate chromosomes, the distribution of the decamer shows symmetrical patterns including inverted repeats. PMID:23841049
NASA Astrophysics Data System (ADS)
Wen, Xueda; Matsuura, Shunji; Ryu, Shinsei
2016-06-01
We develop an approach based on edge theories to calculate the entanglement entropy and related quantities in (2+1)-dimensional topologically ordered phases. Our approach is complementary to, e.g., the existing methods using replica trick and Witten's method of surgery, and applies to a generic spatial manifold of genus g , which can be bipartitioned in an arbitrary way. The effects of fusion and braiding of Wilson lines can be also straightforwardly studied within our framework. By considering a generic superposition of states with different Wilson line configurations, through an interference effect, we can detect, by the entanglement entropy, the topological data of Chern-Simons theories, e.g., the R symbols, monodromy, and topological spins of quasiparticles. Furthermore, by using our method, we calculate other entanglement/correlation measures such as the mutual information and the entanglement negativity. In particular, it is found that the entanglement negativity of two adjacent noncontractible regions on a torus provides a simple way to distinguish Abelian and non-Abelian topological orders.
Hao, Yong; Sun, Xu-dong; Cai, Li-jun; Liu, Yan-de
2012-01-01
Near infrared diffuse reflectance (NIRS) and ultraviolet (UV) spectral analysis were adopted for quantitative determination of octane number and monoaromatics in fuel oil. Partial least squares regression (PLSR) was used for construction of vibrational spectral calibration models. Variables selection strategy based on mutual information (MI) theory was introduced to optimize the models for improving the precision and reducing the complexity. The results indicate that MI-PLSR method can effectively improve the predictive ability of the models and simplify them. For octane number models, the root mean square error of prediction (RMSEP) and the number of calibration variables were reduced from 0.288 and 401 to 0.111 and 112, respectively, and correlation coefficient (R) was improved from 0.985 to 0.998. For monoaromatics models, RMSEP and the number of calibration variables were reduced from 0.753 and 572 to 0.478 and 37, respectively, and R was improved from 0.996 to 0.998. Vibrational spectral analysis combined with MI-PLSR method can be used for quantitative analysis of fuel oil properties, and improve the cost-effectiveness. PMID:22497153
Improving the trust algorithm of information in semantic web
NASA Astrophysics Data System (ADS)
Wan, Zong-bao; Min, Jiang
2012-01-01
With the rapid development of computer networks, especially with the introduction of the Semantic Web perspective, the problem of trust computation in the network has become an important research part of current computer system theoretical. In this paper, according the information properties of the Semantic Web and interact between nodes, the definition semantic trust as content trust of information and the node trust between the nodes of two parts. By Calculate the content of the trust of information and the trust between nodes, then get the final credibility num of information in semantic web. In this paper , we are improve the computation algorithm of the node trust .Finally, stimulations and analyses show that the improved algorithm can effectively improve the trust of information more accurately.
Improving the trust algorithm of information in semantic web
NASA Astrophysics Data System (ADS)
Wan, Zong-Bao; Min, Jiang
2011-12-01
With the rapid development of computer networks, especially with the introduction of the Semantic Web perspective, the problem of trust computation in the network has become an important research part of current computer system theoretical. In this paper, according the information properties of the Semantic Web and interact between nodes, the definition semantic trust as content trust of information and the node trust between the nodes of two parts. By Calculate the content of the trust of information and the trust between nodes, then get the final credibility num of information in semantic web. In this paper , we are improve the computation algorithm of the node trust .Finally, stimulations and analyses show that the improved algorithm can effectively improve the trust of information more accurately.
Imaging for dismantlement verification: information management and analysis algorithms
Seifert, Allen; Miller, Erin A.; Myjak, Mitchell J.; Robinson, Sean M.; Jarman, Kenneth D.; Misner, Alex C.; Pitts, W. Karl; Woodring, Mitchell L.
2010-09-01
The level of detail discernible in imaging techniques has generally excluded them from consideration as verification tools in inspection regimes. An image will almost certainly contain highly sensitive information, and storing a comparison image will almost certainly violate a cardinal principle of information barriers: that no sensitive information be stored in the system. To overcome this problem, some features of the image might be reduced to a few parameters suitable for definition as an attribute. However, this process must be performed with care. Computing the perimeter, area, and intensity of an object, for example, might reveal sensitive information relating to shape, size, and material composition. This paper presents three analysis algorithms that reduce full image information to non-sensitive feature information. Ultimately, the algorithms are intended to provide only a yes/no response verifying the presence of features in the image. We evaluate the algorithms on both their technical performance in image analysis, and their application with and without an explicitly constructed information barrier. The underlying images can be highly detailed, since they are dynamically generated behind the information barrier. We consider the use of active (conventional) radiography alone and in tandem with passive (auto) radiography.
Imaging for dismantlement verification: information management and analysis algorithms
Robinson, Sean M.; Jarman, Kenneth D.; Pitts, W. Karl; Seifert, Allen; Misner, Alex C.; Woodring, Mitchell L.; Myjak, Mitchell J.
2012-01-11
The level of detail discernible in imaging techniques has generally excluded them from consideration as verification tools in inspection regimes. An image will almost certainly contain highly sensitive information, and storing a comparison image will almost certainly violate a cardinal principle of information barriers: that no sensitive information be stored in the system. To overcome this problem, some features of the image might be reduced to a few parameters suitable for definition as an attribute, which must be non-sensitive to be acceptable in an Information Barrier regime. However, this process must be performed with care. Features like the perimeter, area, and intensity of an object, for example, might reveal sensitive information. Any data-reduction technique must provide sufficient information to discriminate a real object from a spoofed or incorrect one, while avoiding disclosure (or storage) of any sensitive object qualities. Ultimately, algorithms are intended to provide only a yes/no response verifying the presence of features in the image. We discuss the utility of imaging for arms control applications and present three image-based verification algorithms in this context. The algorithms reduce full image information to non-sensitive feature information, in a process that is intended to enable verification while eliminating the possibility of image reconstruction. The underlying images can be highly detailed, since they are dynamically generated behind an information barrier. We consider the use of active (conventional) radiography alone and in tandem with passive (auto) radiography. We study these algorithms in terms of technical performance in image analysis and application to an information barrier scheme.
Ossadtchi, Alexei; Pronko, Platon; Baillet, Sylvain; Pflieger, Mark E.; Stroganova, Tatiana
2014-01-01
Spatial component analysis is often used to explore multidimensional time series data whose sources cannot be measured directly. Several methods may be used to decompose the data into a set of spatial components with temporal loadings. Component selection is of crucial importance, and should be supported by objective criteria. In some applications, the use of a well defined component selection criterion may provide for automation of the analysis. In this paper we describe a novel approach for ranking of spatial components calculated from the EEG or MEG data recorded within evoked response paradigm. Our method is called Mutual Information (MI) Spectrum and is based on gauging the amount of MI of spatial component temporal loadings with a synthetically created reference signal. We also describe the appropriate randomization based statistical assessment scheme that can be used for selection of components with statistically significant amount of MI. Using simulated data with realistic trial to trial variations and SNR corresponding to the real recordings we demonstrate the superior performance characteristics of the described MI based measure as compared to a more conventionally used power driven gauge. We also demonstrate the application of the MI Spectrum for the selection of task-related independent components from real MEG data. We show that the MI spectrum allows to identify task-related components reliably in a consistent fashion, yielding stable results even from a small number of trials. We conclude that the proposed method fits naturally the information driven nature of ICA and can be used for routine and automatic ranking of independent components calculated from the functional neuroimaging data collected within event-related paradigms. PMID:24478692
Crossover Improvement for the Genetic Algorithm in Information Retrieval.
ERIC Educational Resources Information Center
Vrajitoru, Dana
1998-01-01
In information retrieval (IR), the aim of genetic algorithms (GA) is to help a system to find, in a huge documents collection, a good reply to a query expressed by the user. Analysis of phenomena seen during the implementation of a GA for IR has led to a new crossover operation, which is introduced and compared to other learning methods.…
Improved motion information-based infrared dim target tracking algorithms
NASA Astrophysics Data System (ADS)
Lei, Liu; Zhijian, Huang
2014-11-01
Accurate and fast tracking of infrared (IR) dim target has very important meaning for infrared precise guidance, early warning, video surveillance, etc. However, under complex backgrounds, such as clutter, varying illumination, and occlusion, the traditional tracking method often converges to a local maximum and loses the real infrared target. To cope with these problems, three improved tracking algorithm based on motion information are proposed in this paper, namely improved mean shift algorithm, improved Optical flow method and improved Particle Filter method. The basic principles and the implementing procedure of these modified algorithms for target tracking are described. Using these algorithms, the experiments on some real-life IR and color images are performed. The whole algorithm implementing processes and results are analyzed, and those algorithms for tracking targets are evaluated from the two aspects of subjective and objective. The results prove that the proposed method has satisfying tracking effectiveness and robustness. Meanwhile, it has high tracking efficiency and can be used for real-time tracking.
A Motion Detection Algorithm Using Local Phase Information
Lazar, Aurel A.; Ukani, Nikul H.; Zhou, Yiyin
2016-01-01
Previous research demonstrated that global phase alone can be used to faithfully represent visual scenes. Here we provide a reconstruction algorithm by using only local phase information. We also demonstrate that local phase alone can be effectively used to detect local motion. The local phase-based motion detector is akin to models employed to detect motion in biological vision, for example, the Reichardt detector. The local phase-based motion detection algorithm introduced here consists of two building blocks. The first building block measures/evaluates the temporal change of the local phase. The temporal derivative of the local phase is shown to exhibit the structure of a second order Volterra kernel with two normalized inputs. We provide an efficient, FFT-based algorithm for implementing the change of the local phase. The second processing building block implements the detector; it compares the maximum of the Radon transform of the local phase derivative with a chosen threshold. We demonstrate examples of applying the local phase-based motion detection algorithm on several video sequences. We also show how the locally detected motion can be used for segmenting moving objects in video scenes and compare our local phase-based algorithm to segmentation achieved with a widely used optic flow algorithm. PMID:26880882
Ren, X; Gao, H; Sharp, G
2015-06-15
Purpose: The delineation of targets and organs-at-risk is a critical step during image-guided radiation therapy, for which manual contouring is the gold standard. However, it is often time-consuming and may suffer from intra- and inter-rater variability. The purpose of this work is to investigate the automated segmentation. Methods: The automatic segmentation here is based on mutual information (MI), with the atlas from Public Domain Database for Computational Anatomy (PDDCA) with manually drawn contours.Using dice coefficient (DC) as the quantitative measure of segmentation accuracy, we perform leave-one-out cross-validations for all PDDCA images sequentially, during which other images are registered to each chosen image and DC is computed between registered contour and ground truth. Meanwhile, six strategies, including MI, are selected to measure the image similarity, with MI to be the best. Then given a target image to be segmented and an atlas, automatic segmentation consists of: (a) the affine registration step for image positioning; (b) the active demons registration method to register the atlas to the target image; (c) the computation of MI values between the deformed atlas and the target image; (d) the weighted image fusion of three deformed atlas images with highest MI values to form the segmented contour. Results: MI was found to be the best among six studied strategies in the sense that it had the highest positive correlation between similarity measure (e.g., MI values) and DC. For automated segmentation, the weighted image fusion of three deformed atlas images with highest MI values provided the highest DC among four proposed strategies. Conclusion: MI has the highest correlation with DC, and therefore is an appropriate choice for post-registration atlas selection in atlas-based segmentation. Xuhua Ren and Hao Gao were partially supported by the NSFC (#11405105), the 973 Program (#2015CB856000) and the Shanghai Pujiang Talent Program (#14PJ1404500)
Dey, Siddharth S.; Xue, Yuhua; Joachimiak, Marcin P.; Friedland, Gregory D.; Burnett, John C.; Zhou, Qiang; Arkin, Adam P.; Schaffer, David V.
2012-01-01
Viral genomes are continually subjected to mutations, and functionally deleterious ones can be rescued by reversion or additional mutations that restore fitness. The error prone nature of HIV-1 replication has resulted in highly diverse viral sequences, and it is not clear how viral proteins such as Tat, which plays a critical role in viral gene expression and replication, retain their complex functions. Although several important amino acid positions in Tat are conserved, we hypothesized that it may also harbor functionally important residues that may not be individually conserved yet appear as correlated pairs, whose analysis could yield new mechanistic insights into Tat function and evolution. To identify such sites, we combined mutual information analysis and experimentation to identify coevolving positions and found that residues 35 and 39 are strongly correlated. Mutation of either residue of this pair into amino acids that appear in numerous viral isolates yields a defective virus; however, simultaneous introduction of both mutations into the heterologous Tat sequence restores gene expression close to wild-type Tat. Furthermore, in contrast to most coevolving protein residues that contribute to the same function, structural modeling and biochemical studies showed that these two residues contribute to two mechanistically distinct steps in gene expression: binding P-TEFb and promoting P-TEFb phosphorylation of the C-terminal domain in RNAPII. Moreover, Tat variants that mimic HIV-1 subtypes B or C at sites 35 and 39 have evolved orthogonal strengths of P-TEFb binding versus RNAPII phosphorylation, suggesting that subtypes have evolved alternate transcriptional strategies to achieve similar gene expression levels. PMID:22253435
Decision making algorithm for development strategy of information systems
NASA Astrophysics Data System (ADS)
Derman, Galyna Y.; Nikitenko, Olena D.; Kotyra, Andrzej; Bazarova, Madina; Kassymkhanova, Dana
2015-12-01
The paper presents algorithm of decision making for development strategy of information systems. The process of development is planned taking into account the internal and external factors of the enterprise which affect the prospects of development of both the information system and the whole enterprise. The initial state of the system must be taken into account. The total risk is the criterion for selecting the strategy. The risk is calculated using statistical and fuzzy data of system's parameters. These data are summarized by means of the function of uncertainty. The software for the realization of the algorithm of decision making on choosing the development strategy of information system is developed and created in this paper.
An Iterative Decoding Algorithm for Fusion of Multimodal Information
NASA Astrophysics Data System (ADS)
Shivappa, Shankar T.; Rao, Bhaskar D.; Trivedi, Mohan M.
2007-12-01
Human activity analysis in an intelligent space is typically based on multimodal informational cues. Use of multiple modalities gives us a lot of advantages. But information fusion from different sources is a problem that has to be addressed. In this paper, we propose an iterative algorithm to fuse information from multimodal sources. We draw inspiration from the theory of turbo codes. We draw an analogy between the redundant parity bits of the constituent codes of a turbo code and the information from different sensors in a multimodal system. A hidden Markov model is used to model the sequence of observations of individual modalities. The decoded state likelihoods from one modality are used as additional information in decoding the states of the other modalities. This procedure is repeated until a certain convergence criterion is met. The resulting iterative algorithm is shown to have lower error rates than the individual models alone. The algorithm is then applied to a real-world problem of speech segmentation using audio and visual cues.
Retaining local image information in gamut mapping algorithms.
Zolliker, Peter; Simon, Klaus
2007-03-01
Our topic is the potential of combining global gamut mapping with spatial methods to retain the percepted local image information in gamut mapping algorithms. The main goal is to recover the original local contrast between neighboring pixels in addition to the usual optimization of preserving lightness, saturation, and global contrast. Special emphasis is placed on avoiding artifacts introduced by the gamut mapping algorithm itself. We present an unsharp masking technique based on an edge-preserving smoothing algorithm allowing to avoid halo artifacts. The good performance of the presented approach is verified by a psycho-visual experiment using newspaper printing as a representative of a small destination gamut application. Furthermore, the improved mapping properties are documented with local mapping histograms. PMID:17357727
Bhowmick, Asmit; Sharma, Sudhir C; Honma, Hallie; Head-Gordon, Teresa
2016-07-28
Side chain entropy and mutual entropy information between residue pairs have been calculated for two de novo designed Kemp eliminase enzymes, KE07 and KE70, and for their most improved versions at the end of laboratory directed evolution (LDE). We find that entropy, not just enthalpy, helped to destabilize the preference for the reactant state complex of the designed enzyme as well as favoring stabilization of the transition state complex for the best LDE enzymes. Furthermore, residues with the highest side chain couplings as measured by mutual information, when experimentally mutated, were found to diminish or annihilate catalytic activity, some of which were far from the active site. In summary, our findings demonstrate how side chain fluctuations and their coupling can be an important design feature for de novo enzymes, and furthermore could be utilized in the computational steps in lieu of or in addition to the LDE steps in future enzyme design projects. PMID:27374812
Alpatov, A. V.; Vikhrov, S. P.; Rybina, N. V.
2015-04-15
The processes of self-organization of the surface structure of hydrogenated amorphous silicon are studied by the methods of fluctuation analysis and average mutual information on the basis of atomic-force-microscopy images of the surface. It is found that all of the structures can be characterized by a correlation vector and represented as a superposition of harmonic components and noise. It is shown that, under variations in the technological parameters of the production of a-Si:H films, the correlation properties of their structure vary as well. As the substrate temperature is increased, the formation of structural irregularities becomes less efficient; in this case, the length of the correlation vector and the degree of structural ordering increase. It is shown that the procedure based on the method of fluctuation analysis in combination with the method of average mutual information provides a means for studying the self-organization processes in any structures on different length scales.
Klus, G T; Song, A; Schick, A; Wahde, M; Szallasi, Z
2001-01-01
Most human tumors are characterized by: (1) an aberrant set of chromosomes, a state termed aneuploidy; (2) an aberrant gene expression pattern; and (3) an aberrant phenotype of uncontrolled growth. One of the goals of cancer research is to establish causative relationships between these three important characteristics. In this paper we were searching for evidence that aneuploidy is a major cause of differential gene expression. We describe how mutual information analysis of cancer-associated gene expression patterns could be exploited to answer this question. In addition to providing general guidelines, we have applied the proposed analysis to a recently published breast cancer-associated gene expression matrix. The results derived from this particular data set provided preliminary evidence that mutual information analysis may become a useful tool to investigate the link between differential gene expression and aneuploidy. PMID:11262960
Yang, Feng; Ding, Mingyue; Zhang, Xuming; Wu, Yi; Hu, Jiani
2013-01-01
Non-rigid multi-modal image registration plays an important role in medical image processing and analysis. Existing image registration methods based on similarity metrics such as mutual information (MI) and sum of squared differences (SSD) cannot achieve either high registration accuracy or high registration efficiency. To address this problem, we propose a novel two phase non-rigid multi-modal image registration method by combining Weber local descriptor (WLD) based similarity metrics with the normalized mutual information (NMI) using the diffeomorphic free-form deformation (FFD) model. The first phase aims at recovering the large deformation component using the WLD based non-local SSD (wldNSSD) or weighted structural similarity (wldWSSIM). Based on the output of the former phase, the second phase is focused on getting accurate transformation parameters related to the small deformation using the NMI. Extensive experiments on T1, T2 and PD weighted MR images demonstrate that the proposed wldNSSD-NMI or wldWSSIM-NMI method outperforms the registration methods based on the NMI, the conditional mutual information (CMI), the SSD on entropy images (ESSD) and the ESSD-NMI in terms of registration accuracy and computation efficiency. PMID:23765270
Artificial Bee Colony Algorithm Based on Information Learning.
Gao, Wei-Feng; Huang, Ling-Ling; Liu, San-Yang; Dai, Cai
2015-12-01
Inspired by the fact that the division of labor and cooperation play extremely important roles in the human history development, this paper develops a novel artificial bee colony algorithm based on information learning (ILABC, for short). In ILABC, at each generation, the whole population is divided into several subpopulations by the clustering partition and the size of subpopulation is dynamically adjusted based on the last search experience, which results in a clear division of labor. Furthermore, the two search mechanisms are designed to facilitate the exchange of information in each subpopulation and between different subpopulations, respectively, which acts as the cooperation. Finally, the comparison results on a number of benchmark functions demonstrate that the proposed method performs competitively and effectively when compared to the selected state-of-the-art algorithms. PMID:25594992
A New Algorithm to Optimize Maximal Information Coefficient.
Chen, Yuan; Zeng, Ying; Luo, Feng; Yuan, Zheming
2016-01-01
The maximal information coefficient (MIC) captures dependences between paired variables, including both functional and non-functional relationships. In this paper, we develop a new method, ChiMIC, to calculate the MIC values. The ChiMIC algorithm uses the chi-square test to terminate grid optimization and then removes the restriction of maximal grid size limitation of original ApproxMaxMI algorithm. Computational experiments show that ChiMIC algorithm can maintain same MIC values for noiseless functional relationships, but gives much smaller MIC values for independent variables. For noise functional relationship, the ChiMIC algorithm can reach the optimal partition much faster. Furthermore, the MCN values based on MIC calculated by ChiMIC can capture the complexity of functional relationships in a better way, and the statistical powers of MIC calculated by ChiMIC are higher than those calculated by ApproxMaxMI. Moreover, the computational costs of ChiMIC are much less than those of ApproxMaxMI. We apply the MIC values tofeature selection and obtain better classification accuracy using features selected by the MIC values from ChiMIC. PMID:27333001
A New Algorithm to Optimize Maximal Information Coefficient
Luo, Feng; Yuan, Zheming
2016-01-01
The maximal information coefficient (MIC) captures dependences between paired variables, including both functional and non-functional relationships. In this paper, we develop a new method, ChiMIC, to calculate the MIC values. The ChiMIC algorithm uses the chi-square test to terminate grid optimization and then removes the restriction of maximal grid size limitation of original ApproxMaxMI algorithm. Computational experiments show that ChiMIC algorithm can maintain same MIC values for noiseless functional relationships, but gives much smaller MIC values for independent variables. For noise functional relationship, the ChiMIC algorithm can reach the optimal partition much faster. Furthermore, the MCN values based on MIC calculated by ChiMIC can capture the complexity of functional relationships in a better way, and the statistical powers of MIC calculated by ChiMIC are higher than those calculated by ApproxMaxMI. Moreover, the computational costs of ChiMIC are much less than those of ApproxMaxMI. We apply the MIC values tofeature selection and obtain better classification accuracy using features selected by the MIC values from ChiMIC. PMID:27333001
Optical tomographic memories: algorithms for the efficient information readout
NASA Astrophysics Data System (ADS)
Pantelic, Dejan V.
1990-07-01
Tomographic alogithms are modified in order to reconstruct the inf ormation previously stored by focusing laser radiation in a volume of photosensitive media. Apriori information about the position of bits of inf ormation is used. 1. THE PRINCIPLES OF TOMOGRAPHIC MEMORIES Tomographic principles can be used to store and reconstruct the inf ormation artificially stored in a bulk of a photosensitive media 1 The information is stored by changing some characteristics of a memory material (e. g. refractive index). Radiation from the two independent light sources (e. g. lasers) is f ocused inside the memory material. In this way the intensity of the light is above the threshold only in the localized point where the light rays intersect. By scanning the material the information can be stored in binary or nary format. When the information is stored it can be read by tomographic methods. However the situation is quite different from the classical tomographic problem. Here a lot of apriori information is present regarding the p0- sitions of the bits of information profile representing single bit and a mode of operation (binary or n-ary). 2. ALGORITHMS FOR THE READOUT OF THE TOMOGRAPHIC MEMORIES Apriori information enables efficient reconstruction of the memory contents. In this paper a few methods for the information readout together with the simulation results will be presented. Special attention will be given to the noise considerations. Two different
Network algorithms for information analysis using the Titan Toolkit.
McLendon, William Clarence, III; Baumes, Jeffrey; Wilson, Andrew T.; Wylie, Brian Neil; Shead, Timothy M.
2010-07-01
The analysis of networked activities is dramatically more challenging than many traditional kinds of analysis. A network is defined by a set of entities (people, organizations, banks, computers, etc.) linked by various types of relationships. These entities and relationships are often uninteresting alone, and only become significant in aggregate. The analysis and visualization of these networks is one of the driving factors behind the creation of the Titan Toolkit. Given the broad set of problem domains and the wide ranging databases in use by the information analysis community, the Titan Toolkit's flexible, component based pipeline provides an excellent platform for constructing specific combinations of network algorithms and visualizations.
Information dynamics algorithm for detecting communities in networks
NASA Astrophysics Data System (ADS)
Massaro, Emanuele; Bagnoli, Franco; Guazzini, Andrea; Lió, Pietro
2012-11-01
The problem of community detection is relevant in many scientific disciplines, from social science to statistical physics. Given the impact of community detection in many areas, such as psychology and social sciences, we have addressed the issue of modifying existing well performing algorithms by incorporating elements of the domain application fields, i.e. domain-inspired. We have focused on a psychology and social network-inspired approach which may be useful for further strengthening the link between social network studies and mathematics of community detection. Here we introduce a community-detection algorithm derived from the van Dongen's Markov Cluster algorithm (MCL) method [4] by considering networks' nodes as agents capable to take decisions. In this framework we have introduced a memory factor to mimic a typical human behavior such as the oblivion effect. The method is based on information diffusion and it includes a non-linear processing phase. We test our method on two classical community benchmark and on computer generated networks with known community structure. Our approach has three important features: the capacity of detecting overlapping communities, the capability of identifying communities from an individual point of view and the fine tuning the community detectability with respect to prior knowledge of the data. Finally we discuss how to use a Shannon entropy measure for parameter estimation in complex networks.
Information theoretic methods for image processing algorithm optimization
NASA Astrophysics Data System (ADS)
Prokushkin, Sergey F.; Galil, Erez
2015-01-01
Modern image processing pipelines (e.g., those used in digital cameras) are full of advanced, highly adaptive filters that often have a large number of tunable parameters (sometimes > 100). This makes the calibration procedure for these filters very complex, and the optimal results barely achievable in the manual calibration; thus an automated approach is a must. We will discuss an information theory based metric for evaluation of algorithm adaptive characteristics ("adaptivity criterion") using noise reduction algorithms as an example. The method allows finding an "orthogonal decomposition" of the filter parameter space into the "filter adaptivity" and "filter strength" directions. This metric can be used as a cost function in automatic filter optimization. Since it is a measure of a physical "information restoration" rather than perceived image quality, it helps to reduce the set of the filter parameters to a smaller subset that is easier for a human operator to tune and achieve a better subjective image quality. With appropriate adjustments, the criterion can be used for assessment of the whole imaging system (sensor plus post-processing).
An Algorithm Combining for Objective Prediction with Subjective Forecast Information
NASA Astrophysics Data System (ADS)
Choi, JunTae; Kim, SooHyun
2016-04-01
As direct or post-processed output from numerical weather prediction (NWP) models has begun to show acceptable performance compared with the predictions of human forecasters, many national weather centers have become interested in automatic forecasting systems based on NWP products alone, without intervention from human forecasters. The Korea Meteorological Administration (KMA) is now developing an automatic forecasting system for dry variables. The forecasts are automatically generated from NWP predictions using a post processing model (MOS). However, MOS cannot always produce acceptable predictions, and sometimes its predictions are rejected by human forecasters. In such cases, a human forecaster should manually modify the prediction consistently at points surrounding their corrections, using some kind of smart tool to incorporate the forecaster's opinion. This study introduces an algorithm to revise MOS predictions by adding a forecaster's subjective forecast information at neighbouring points. A statistical relation between two forecast points - a neighbouring point and a dependent point - was derived for the difference between a MOS prediction and that of a human forecaster. If the MOS prediction at a neighbouring point is updated by a human forecaster, the value at a dependent point is modified using a statistical relationship based on linear regression, with parameters obtained from a one-year dataset of MOS predictions and official forecast data issued by KMA. The best sets of neighbouring points and dependent point are statistically selected. According to verification, the RMSE of temperature predictions produced by the new algorithm was slightly lower than that of the original MOS predictions, and close to the RMSE of subjective forecasts. For wind speed and relative humidity, the new algorithm outperformed human forecasters.
Thermodynamic cost of computation, algorithmic complexity and the information metric
NASA Technical Reports Server (NTRS)
Zurek, W. H.
1989-01-01
Algorithmic complexity is discussed as a computational counterpart to the second law of thermodynamics. It is shown that algorithmic complexity, which is a measure of randomness, sets limits on the thermodynamic cost of computations and casts a new light on the limitations of Maxwell's demon. Algorithmic complexity can also be used to define distance between binary strings.
The accurate reconstruction of gene regulatory networks from large scale molecular profile datasets represents one of the grand challenges of Systems Biology. The Algorithm for the Reconstruction of Accurate Cellular Networks (ARACNe) represents one of the most effective tools to accomplish this goal. However, the initial Fixed Bandwidth (FB) implementation is both inefficient and unable to deal with sample sets providing largely uneven coverage of the probability density space.
On Distribution Reduction and Algorithm Implementation in Inconsistent Ordered Information Systems
Zhang, Yanqin
2014-01-01
As one part of our work in ordered information systems, distribution reduction is studied in inconsistent ordered information systems (OISs). Some important properties on distribution reduction are studied and discussed. The dominance matrix is restated for reduction acquisition in dominance relations based information systems. Matrix algorithm for distribution reduction acquisition is stepped. And program is implemented by the algorithm. The approach provides an effective tool for the theoretical research and the applications for ordered information systems in practices. For more detailed and valid illustrations, cases are employed to explain and verify the algorithm and the program which shows the effectiveness of the algorithm in complicated information systems. PMID:25258721
An Introduction to Genetic Algorithms and to Their Use in Information Retrieval.
ERIC Educational Resources Information Center
Jones, Gareth; And Others
1994-01-01
Genetic algorithms, a class of nondeterministic algorithms in which the role of chance makes the precise nature of a solution impossible to guarantee, seem to be well suited to combinatorial-optimization problems in information retrieval. Provides an introduction to techniques and characteristics of genetic algorithms and illustrates their…
Mutual coupling, channel model, and BER for curvilinear antenna arrays
NASA Astrophysics Data System (ADS)
Huang, Zhiyong
interferers, Doppler spread and convergence are investigated. The tracking mode is introduced to the adaptive array system, and it further improves the BER. The benefit of using faster data rate (wider bandwidth) is discussed. In order to have better performance in a 3D space, the geometries of uniform spherical array (USAs) are presented and different configurations of USAs are discussed. The LMS algorithm based on temporal a priori information is applied to UCAs and USAs to beamform the patterns. Their performances are compared based on simulation results. Based on the analytical and simulation results, it can be concluded that mutual coupling slightly influences the performance of the adaptive array in communication systems. In addition, arrays with curvilinear geometries perform well in AWGN and fading channels.
NASA Astrophysics Data System (ADS)
Chen, Lei; Li, Dehua; Yang, Jie
2007-12-01
Constructing virtual international strategy environment needs many kinds of information, such as economy, politic, military, diploma, culture, science, etc. So it is very important to build an information auto-extract, classification, recombination and analysis management system with high efficiency as the foundation and component of military strategy hall. This paper firstly use improved Boost algorithm to classify obtained initial information, then use a strategy intelligence extract algorithm to extract strategy intelligence from initial information to help strategist to analysis information.
NASA Astrophysics Data System (ADS)
A. AL-Salhi, Yahya E.; Lu, Songfeng
2016-04-01
Quantum steganography can solve some problems that are considered inefficient in image information concealing. It researches on Quantum image information concealing to have been widely exploited in recent years. Quantum image information concealing can be categorized into quantum image digital blocking, quantum image stereography, anonymity and other branches. Least significant bit (LSB) information concealing plays vital roles in the classical world because many image information concealing algorithms are designed based on it. Firstly, based on the novel enhanced quantum representation (NEQR), image uniform blocks clustering around the concrete the least significant Qu-block (LSQB) information concealing algorithm for quantum image steganography is presented. Secondly, a clustering algorithm is proposed to optimize the concealment of important data. Finally, we used Con-Steg algorithm to conceal the clustered image blocks. Information concealing located on the Fourier domain of an image can achieve the security of image information, thus we further discuss the Fourier domain LSQu-block information concealing algorithm for quantum image based on Quantum Fourier Transforms. In our algorithms, the corresponding unitary Transformations are designed to realize the aim of concealing the secret information to the least significant Qu-block representing color of the quantum cover image. Finally, the procedures of extracting the secret information are illustrated. Quantum image LSQu-block image information concealing algorithm can be applied in many fields according to different needs.
NASA Astrophysics Data System (ADS)
A. AL-Salhi, Yahya E.; Lu, Songfeng
2016-08-01
Quantum steganography can solve some problems that are considered inefficient in image information concealing. It researches on Quantum image information concealing to have been widely exploited in recent years. Quantum image information concealing can be categorized into quantum image digital blocking, quantum image stereography, anonymity and other branches. Least significant bit (LSB) information concealing plays vital roles in the classical world because many image information concealing algorithms are designed based on it. Firstly, based on the novel enhanced quantum representation (NEQR), image uniform blocks clustering around the concrete the least significant Qu-block (LSQB) information concealing algorithm for quantum image steganography is presented. Secondly, a clustering algorithm is proposed to optimize the concealment of important data. Finally, we used Con-Steg algorithm to conceal the clustered image blocks. Information concealing located on the Fourier domain of an image can achieve the security of image information, thus we further discuss the Fourier domain LSQu-block information concealing algorithm for quantum image based on Quantum Fourier Transforms. In our algorithms, the corresponding unitary Transformations are designed to realize the aim of concealing the secret information to the least significant Qu-block representing color of the quantum cover image. Finally, the procedures of extracting the secret information are illustrated. Quantum image LSQu-block image information concealing algorithm can be applied in many fields according to different needs.
Algorithmic complexity of a protein
NASA Astrophysics Data System (ADS)
Dewey, T. Gregory
1996-07-01
The information contained in a protein's amino acid sequence dictates its three-dimensional structure. To quantitate the transfer of information that occurs in the protein folding process, the Kolmogorov information entropy or algorithmic complexity of the protein structure is investigated. The algorithmic complexity of an object provides a means of quantitating its information content. Recent results have indicated that the algorithmic complexity of microstates of certain statistical mechanical systems can be estimated from the thermodynamic entropy. In the present work, it is shown that the algorithmic complexity of a protein is given by its configurational entropy. Using this result, a quantitative estimate of the information content of a protein's structure is made and is compared to the information content of the sequence. Additionally, the mutual information between sequence and structure is determined. It is seen that virtually all the information contained in the protein structure is shared with the sequence.
Star pattern recognition algorithm aided by inertial information
NASA Astrophysics Data System (ADS)
Liu, Bao; Wang, Ke-dong; Zhang, Chao
2011-08-01
Star pattern recognition is one of the key problems of the celestial navigation. The traditional star pattern recognition approaches, such as the triangle algorithm and the star angular distance algorithm, are a kind of all-sky matching method whose recognition speed is slow and recognition success rate is not high. Therefore, the real time and reliability of CNS (Celestial Navigation System) is reduced to some extent, especially for the maneuvering spacecraft. However, if the direction of the camera optical axis can be estimated by other navigation systems such as INS (Inertial Navigation System), the star pattern recognition can be fulfilled in the vicinity of the estimated direction of the optical axis. The benefits of the INS-aided star pattern recognition algorithm include at least the improved matching speed and the improved success rate. In this paper, the direction of the camera optical axis, the local matching sky, and the projection of stars on the image plane are estimated by the aiding of INS firstly. Then, the local star catalog for the star pattern recognition is established in real time dynamically. The star images extracted in the camera plane are matched in the local sky. Compared to the traditional all-sky star pattern recognition algorithms, the memory of storing the star catalog is reduced significantly. Finally, the INS-aided star pattern recognition algorithm is validated by simulations. The results of simulations show that the algorithm's computation time is reduced sharply and its matching success rate is improved greatly.
Algorithm for shortest path search in Geographic Information Systems by using reduced graphs.
Rodríguez-Puente, Rafael; Lazo-Cortés, Manuel S
2013-01-01
The use of Geographic Information Systems has increased considerably since the eighties and nineties. As one of their most demanding applications we can mention shortest paths search. Several studies about shortest path search show the feasibility of using graphs for this purpose. Dijkstra's algorithm is one of the classic shortest path search algorithms. This algorithm is not well suited for shortest path search in large graphs. This is the reason why various modifications to Dijkstra's algorithm have been proposed by several authors using heuristics to reduce the run time of shortest path search. One of the most used heuristic algorithms is the A* algorithm, the main goal is to reduce the run time by reducing the search space. This article proposes a modification of Dijkstra's shortest path search algorithm in reduced graphs. It shows that the cost of the path found in this work, is equal to the cost of the path found using Dijkstra's algorithm in the original graph. The results of finding the shortest path, applying the proposed algorithm, Dijkstra's algorithm and A* algorithm, are compared. This comparison shows that, by applying the approach proposed, it is possible to obtain the optimal path in a similar or even in less time than when using heuristic algorithms. PMID:24010024
NASA Astrophysics Data System (ADS)
Núñez-Acosta, Elisa; Lerma, Claudia; Márquez, Manlio F.; José, Marco V.
2012-02-01
Herein we introduce the Mutual Information Function (MIF) as a mathematical method to analyze ventricular bigeminy in certain pathological conditions of the heart known to be associated with frequent ventricular arrhythmias. In particular, we show that the MIF is sensitive enough to detect the bigeminy pattern in symbolic series from patients with Andersen-Tawil syndrome as well as in a group of patients from the Sudden Cardiac Death Holter Databases. The results confirm that MIF is an adequate method to detect the autocorrelation between the appearance of sinus and ventricular premature beats resulting in a bigeminy pattern. It is also shown that MIF reflects the bigeminy patterns as a function of the percentage of ventricular premature beats present in the symbolic series and also as a function of the percentage of bigeminy. The MIF was also useful to establish a consistent difference in the bigeminy pattern related to the diurnal and nocturnal periods presumably associated to the circadian rhythm of the heart. Understanding of the ventricular bigeminy patterns throughout 24-hours could provide some insights into the pathogenesis of ventricular tachyarrhythmias in these pathological conditions.
NASA Astrophysics Data System (ADS)
Adamowski, J. F.; Quilty, J.; Khalil, B.; Rathinasamy, M.
2014-12-01
This paper explores forecasting short-term urban water demand (UWD) (using only historical records) through a variety of machine learning techniques coupled with a novel input variable selection (IVS) procedure. The proposed IVS technique termed, bootstrap rank-ordered conditional mutual information for real-valued signals (brCMIr), is multivariate, nonlinear, nonparametric, and probabilistic. The brCMIr method was tested in a case study using water demand time series for two urban water supply system pressure zones in Ottawa, Canada to select the most important historical records for use with each machine learning technique in order to generate forecasts of average and peak UWD for the respective pressure zones at lead times of 1, 3, and 7 days ahead. All lead time forecasts are computed using Artificial Neural Networks (ANN) as the base model, and are compared with Least Squares Support Vector Regression (LSSVR), as well as a novel machine learning method for UWD forecasting: the Extreme Learning Machine (ELM). Results from one-way analysis of variance (ANOVA) and Tukey Honesty Significance Difference (HSD) tests indicate that the LSSVR and ELM models are the best machine learning techniques to pair with brCMIr. However, ELM has significant computational advantages over LSSVR (and ANN) and provides a new and promising technique to explore in UWD forecasting.
Amanpour, Behzad; Erfanian, Abbas
2013-01-01
An important issue in designing a practical brain-computer interface (BCI) is the selection of mental tasks to be imagined. Different types of mental tasks have been used in BCI including left, right, foot, and tongue motor imageries. However, the mental tasks are different from the actions to be controlled by the BCI. It is desirable to select a mental task to be consistent with the desired action to be performed by BCI. In this paper, we investigated the detecting the imagination of the hand grasping, hand opening, and hand reaching in one hand using electroencephalographic (EEG) signals. The results show that the ERD/ERS patterns, associated with the imagination of hand grasping, opening, and reaching are different. For classification of brain signals associated with these mental tasks and feature extraction, a method based on wavelet packet, regularized common spatial pattern (CSP), and mutual information is proposed. The results of an offline analysis on five subjects show that the two-class mental tasks can be classified with an average accuracy of 77.6% using proposed method. In addition, we examine the proposed method on datasets IVa from BCI Competition III and IIa from BCI Competition IV. PMID:24110165
Developing Information Power Grid Based Algorithms and Software
NASA Technical Reports Server (NTRS)
Dongarra, Jack
1998-01-01
This exploratory study initiated our effort to understand performance modeling on parallel systems. The basic goal of performance modeling is to understand and predict the performance of a computer program or set of programs on a computer system. Performance modeling has numerous applications, including evaluation of algorithms, optimization of code implementations, parallel library development, comparison of system architectures, parallel system design, and procurement of new systems. Our work lays the basis for the construction of parallel libraries that allow for the reconstruction of application codes on several distinct architectures so as to assure performance portability. Following our strategy, once the requirements of applications are well understood, one can then construct a library in a layered fashion. The top level of this library will consist of architecture-independent geometric, numerical, and symbolic algorithms that are needed by the sample of applications. These routines should be written in a language that is portable across the targeted architectures.
A Survey of Stemming Algorithms in Information Retrieval
ERIC Educational Resources Information Center
Moral, Cristian; de Antonio, Angélica; Imbert, Ricardo; Ramírez, Jaime
2014-01-01
Background: During the last fifty years, improved information retrieval techniques have become necessary because of the huge amount of information people have available, which continues to increase rapidly due to the use of new technologies and the Internet. Stemming is one of the processes that can improve information retrieval in terms of…
Infrared image non-rigid registration based on regional information entropy demons algorithm
NASA Astrophysics Data System (ADS)
Lu, Chaoliang; Ma, Lihua; Yu, Ming; Cui, Shumin; Wu, Qingrong
2015-02-01
Infrared imaging fault detection which is treated as an ideal, non-contact, non-destructive testing method is applied to the circuit board fault detection. Since Infrared images obtained by handheld infrared camera with wide-angle lens have both rigid and non-rigid deformations. To solve this problem, a new demons algorithm based on regional information entropy was proposed. The new method overcame the shortcomings of traditional demons algorithm that was sensitive to the intensity. First, the information entropy image was gotten by computing regional information entropy of the image. Then, the deformation between the two images was calculated that was the same as demons algorithm. Experimental results demonstrated that the proposed algorithm has better robustness in intensity inconsistent images registration compared with the traditional demons algorithm. Achieving accurate registration between intensity inconsistent infrared images provided strong support for the temperature contrast.
Binzel, R.P. )
1989-11-01
Since 1985, planetary astronomers have been working to take advantage of a once-per-century apparent alignment between Pluto and its satellite, Charon, which has allowed mutual occultation and transit events to be observed. There events, which will cease in 1990, have permitted the first precise determinations of their individual radii, densities, and surface compositions. In addition, information on their surface albedo distributions can be obtained.
ERIC Educational Resources Information Center
Chen, Hsinchun
1995-01-01
Presents an overview of artificial-intelligence-based inductive learning techniques and their use in information science research. Three methods are discussed: the connectionist Hopfield network; the symbolic ID3/ID5R; evolution-based genetic algorithms. The knowledge representations and algorithms of these methods are examined in the context of…
Evolution of mutualism between species
Post, W.M.; Travis, C.C.; DeAngelis, D.L.
1980-01-01
Recent theoretical work on mutualism, the interaction between species populations that is mutually beneficial, is reviewed. Several ecological facts that should be addressed in the construction of dynamic models for mutualism are examined. Basic terminology is clarified. (PSB)
Behavioral Ecology: Manipulative Mutualism.
Hughes, David P
2015-09-21
A new study reveals that an apparent mutualism between lycaenid caterpillars and their attendant ants may not be all it seems, as the caterpillars produce secretions that modify the brains and behavior of their attendant ants. PMID:26394105
Non-algorithmic access to calendar information in a calendar calculator with autism.
Mottron, L; Lemmens, K; Gagnon, L; Seron, X
2006-02-01
The possible use of a calendar algorithm was assessed in DBC, an autistic "savant" of normal measured intelligence. Testing of all the dates in a year revealed a random distribution of errors. Re-testing DBC on the same dates one year later shows that his errors were not stable across time. Finally, DBC was able to answer "reversed" questions that cannot be solved by a classical algorithm. These findings favor a non-algorithmic retrieval of calendar information. It is proposed that multidirectional, non-hierarchical retrieval of information, and solving problems in a non-algorithmic way, are involved in savant performances. The possible role of a functional rededication of low-level perceptual systems to the processing of symbolic information in savants is discussed. PMID:16453069
Representing Uncertain Geographical Information with Algorithmic Map Caricatures
NASA Astrophysics Data System (ADS)
Brunsdon, Chris
2016-04-01
A great deal of geographical information - including the results ion data analysis - is imprecise in some way. For example the the results of geostatistical interpolation should consist not only of point estimates of the value of some quantity at points in space, but also of confidence intervals or standard errors of these estimators. Similarly, mappings of contour lines derived form such interpolations will also be characterised by uncertainty. However, most computerized cartography tools are designed to provide 'crisp' representations of geographical information, such as sharply drawn lines, or clearly delineated areas. In this talk, the use of 'fuzzy' or 'sketchy' cartographic tools will be demonstrated - where maps have a hand-drawn appearance and the degree of 'roughness' and other related characteristics can be used to convey the degree of uncertainty associated with estimated quantities being mapped. The tools used to do this are available as an R package, which will be described in the talk.
A Lip Extraction Algorithm by Using Color Information Considering Obscurity
NASA Astrophysics Data System (ADS)
Shirasawa, Yoichi; Nishida, Makoto
This paper proposes a method for extracting lip shape and its location from sequential facial images by using color information. The proposed method has no need of extra information on a position nor a form in advance. It is also carried out without special conditions such as lipstick or lighting. Psychometric quantities of a metric hue angle, a metric hue difference and a rectangular coordinates, which are defined in CIE 1976 L*a*b* color space, are used for the extraction. The method employs fuzzy reasoning in order to consider obscurity in image data such as shade on the face. The experimental result indicate the effectiveness of the proposed method; 100 percent of facial images data was estimated a lip’s position, and about 94 percent of facial images data was extracted its shape.
Developing Information Power Grid Based Algorithms and Software
NASA Technical Reports Server (NTRS)
Dongarra, Jack
1998-01-01
This was an exploratory study to enhance our understanding of problems involved in developing large scale applications in a heterogeneous distributed environment. It is likely that the large scale applications of the future will be built by coupling specialized computational modules together. For example, efforts now exist to couple ocean and atmospheric prediction codes to simulate a more complete climate system. These two applications differ in many respects. They have different grids, the data is in different unit systems and the algorithms for inte,-rating in time are different. In addition the code for each application is likely to have been developed on different architectures and tend to have poor performance when run on an architecture for which the code was not designed, if it runs at all. Architectural differences may also induce differences in data representation which effect precision and convergence criteria as well as data transfer issues. In order to couple such dissimilar codes some form of translation must be present. This translation should be able to handle interpolation from one grid to another as well as construction of the correct data field in the correct units from available data. Even if a code is to be developed from scratch, a modular approach will likely be followed in that standard scientific packages will be used to do the more mundane tasks such as linear algebra or Fourier transform operations. This approach allows the developers to concentrate on their science rather than becoming experts in linear algebra or signal processing. Problems associated with this development approach include difficulties associated with data extraction and translation from one module to another, module performance on different nodal architectures, and others. In addition to these data and software issues there exists operational issues such as platform stability and resource management.
Feature weighted naïve Bayes algorithm for information retrieval of enterprise systems
NASA Astrophysics Data System (ADS)
Wang, Li; Ji, Ping; Qi, Jing; Shan, Siqing; Bi, Zhuming; Deng, Weiguo; Zhang, Naijing
2014-01-01
Automated information retrieval is critical for enterprise information systems to acquire knowledge from the vast amount of data sets. One challenge in information retrieval is text classification. Current practices rely heavily on the classical naïve Bayes algorithm due to its simplicity and robustness. However, results from this algorithm are not always satisfactory. In this article, the limitations of the naïve Bayes algorithm are discussed, and it is found that the assumption on the independence of terms is the main reason for an unsatisfactory classification in many real-world applications. To overcome the limitations, the dependent factors are considered by integrating a term frequency-inverse document frequency (TF-IDF) weighting algorithm in the naïve Bayes classification. Moreover, the TF-IDF algorithm itself is improved so that both frequencies and distribution information are taken into consideration. To illustrate the effectiveness of the proposed method, two simulation experiments were conducted, and the comparisons with other classification methods have shown that the proposed method has outperformed other existing algorithms in terms of precision and index recall rate.
A Selective Encryption Algorithm Based on AES for Medical Information
Oh, Ju-Young; Chon, Ki-Hwan
2010-01-01
Objectives The transmission of medical information is currently a daily routine. Medical information needs efficient, robust and secure encryption modes, but cryptography is primarily a computationally intensive process. Towards this direction, we design a selective encryption scheme for critical data transmission. Methods We expand the advandced encrytion stanard (AES)-Rijndael with five criteria: the first is the compression of plain data, the second is the variable size of the block, the third is the selectable round, the fourth is the optimization of software implementation and the fifth is the selective function of the whole routine. We have tested our selective encryption scheme by C++ and it was compiled with Code::Blocks using a MinGW GCC compiler. Results The experimental results showed that our selective encryption scheme achieves a faster execution speed of encryption/decryption. In future work, we intend to use resource optimization to enhance the round operations, such as SubByte/InvSubByte, by exploiting similarities between encryption and decryption. Conclusions As encryption schemes become more widely used, the concept of hardware and software co-design is also a growing new area of interest. PMID:21818420
A General Algorithm for Reusing Krylov Subspace Information. I. Unsteady Navier-Stokes
NASA Technical Reports Server (NTRS)
Carpenter, Mark H.; Vuik, C.; Lucas, Peter; vanGijzen, Martin; Bijl, Hester
2010-01-01
A general algorithm is developed that reuses available information to accelerate the iterative convergence of linear systems with multiple right-hand sides A x = b (sup i), which are commonly encountered in steady or unsteady simulations of nonlinear equations. The algorithm is based on the classical GMRES algorithm with eigenvector enrichment but also includes a Galerkin projection preprocessing step and several novel Krylov subspace reuse strategies. The new approach is applied to a set of test problems, including an unsteady turbulent airfoil, and is shown in some cases to provide significant improvement in computational efficiency relative to baseline approaches.
Algorithmic information theory and the hidden variable question
NASA Astrophysics Data System (ADS)
Fuchs, Christopher
1992-02-01
The admissibility of certain nonlocal hidden-variable theories are explained via information theory. Consider a pair of Stern-Gerlach devices with fixed nonparallel orientations that periodically perform spin measurements on identically prepared pairs of electrons in the singlet spin state. Suppose the outcomes are recorded as binary strings l and r (with l sub n and r sub n denoting their n-length prefixes). The hidden-variable theories considered here require that there exists a recursive function which may be used to transform l sub n into r sub n for any n. This note demonstrates that such a theory cannot reproduce all the statistical predictions of quantum mechanics. Specifically, consider an ensemble of outcome pairs (l,r). From the associated probability measure, the Shannon entropies H sub n and H bar sub n for strings l sub n and pairs (l sub n, r sub n) may be formed. It is shown that such a theory requires that the absolute value of H bar sub n - H sub n be bounded - contrasting the quantum mechanical prediction that it grow with n.
Algorithmic information theory and the hidden variable question
NASA Technical Reports Server (NTRS)
Fuchs, Christopher
1992-01-01
The admissibility of certain nonlocal hidden-variable theories are explained via information theory. Consider a pair of Stern-Gerlach devices with fixed nonparallel orientations that periodically perform spin measurements on identically prepared pairs of electrons in the singlet spin state. Suppose the outcomes are recorded as binary strings l and r (with l sub n and r sub n denoting their n-length prefixes). The hidden-variable theories considered here require that there exists a recursive function which may be used to transform l sub n into r sub n for any n. This note demonstrates that such a theory cannot reproduce all the statistical predictions of quantum mechanics. Specifically, consider an ensemble of outcome pairs (l,r). From the associated probability measure, the Shannon entropies H sub n and H bar sub n for strings l sub n and pairs (l sub n, r sub n) may be formed. It is shown that such a theory requires that the absolute value of H bar sub n - H sub n be bounded - contrasting the quantum mechanical prediction that it grow with n.
Deciphering the Minimal Algorithm for Development and Information-genesis
NASA Astrophysics Data System (ADS)
Li, Zhiyuan; Tang, Chao; Li, Hao
During development, cells with identical genomes acquires different fates in a highly organized manner. In order to decipher the principles underlining development, we used C.elegans as the model organism. Based on a large set of microscopy imaging, we first constructed a ``standard worm'' in silico: from the single zygotic cell to about 500 cell stage, the lineage, position, cell-cell contact and gene expression dynamics are quantified for each cell in order to investigate principles underlining these intensive data. Next, we reverse-engineered the possible gene-gene/cell-cell interaction rules that are capable of running a dynamic model recapitulating the early fate decisions during C.elegans development. we further formulized the C.elegans embryogenesis in the language of information genesis. Analysis towards data and model uncovered the global landscape of development in the cell fate space, suggested possible gene regulatory architectures and cell signaling processes, revealed diversity and robustness as the essential trade-offs in development, and demonstrated general strategies in building multicellular organisms.
Mutually Exclusive, Complementary, or . . .
ERIC Educational Resources Information Center
Schloemer, Cathy G.
2016-01-01
Whether students are beginning their study of probability or are well into it, distinctions between complementary sets and mutually exclusive sets can be confusing. Cathy Schloemer writes in this article that for years she used typical classroom examples but was not happy with the student engagement or the level of understanding they produced.…
ERIC Educational Resources Information Center
Siskin, Leslie Santee
2016-01-01
Building on an expanded concept of mutual adaptation, this chapter explores a distinctive and successful aspect of International Baccalaureate's effort to scale up, as they moved to expand their programs and support services in Title I schools. Based on a three-year, mixed-methods study, it offers a case where we see not only local adaptations…
Robust Blind Learning Algorithm for Nonlinear Equalization Using Input Decision Information.
Xu, Lu; Huang, Defeng David; Guo, Yingjie Jay
2015-12-01
In this paper, we propose a new blind learning algorithm, namely, the Benveniste-Goursat input-output decision (BG-IOD), to enhance the convergence performance of neural network-based equalizers for nonlinear channel equalization. In contrast to conventional blind learning algorithms, where only the output of the equalizer is employed for updating system parameters, the BG-IOD exploits a new type of extra information, the input decision information obtained from the input of the equalizer, to mitigate the influence of the nonlinear equalizer structure on parameters learning, thereby leading to improved convergence performance. We prove that, with the input decision information, a desirable convergence capability that the output symbol error rate (SER) is always less than the input SER if the input SER is below a threshold, can be achieved. Then, the BG soft-switching technique is employed to combine the merits of both input and output decision information, where the former is used to guarantee SER convergence and the latter is to improve SER performance. Simulation results show that the proposed algorithm outperforms conventional blind learning algorithms, such as stochastic quadratic distance and dual mode constant modulus algorithm, in terms of both convergence performance and SER performance, for nonlinear equalization. PMID:25706894
NASA Astrophysics Data System (ADS)
Wu, Qiong; Wang, Jihua; Wang, Cheng; Xu, Tongyu
2016-09-01
Genetic algorithm (GA) has a significant effect in the band optimization selection of Partial Least Squares (PLS) correction model. Application of genetic algorithm in selection of characteristic bands can achieve the optimal solution more rapidly, effectively improve measurement accuracy and reduce variables used for modeling. In this study, genetic algorithm as a module conducted band selection for the application of hyperspectral imaging in nondestructive testing of corn seedling leaves, and GA-PLS model was established. In addition, PLS quantitative model of full spectrum and experienced-spectrum region were established in order to suggest the feasibility of genetic algorithm optimizing wave bands, and model robustness was evaluated. There were 12 characteristic bands selected by genetic algorithm. With reflectance values of corn seedling component information at spectral characteristic wavelengths corresponding to 12 characteristic bands as variables, a model about SPAD values of corn leaves acquired was established by PLS, and modeling results showed r = 0.7825. The model results were better than those of PLS model established in full spectrum and experience-based selected bands. The results suggested that genetic algorithm can be used for data optimization and screening before establishing the corn seedling component information model by PLS method and effectively increase measurement accuracy and greatly reduce variables used for modeling.