Bayesian cloud detection for MERIS, AATSR, and their combination

NASA Astrophysics Data System (ADS)

Hollstein, A.; Fischer, J.; Carbajal Henken, C.; Preusker, R.

2015-04-01

A broad range of different of Bayesian cloud detection schemes is applied to measurements from the Medium Resolution Imaging Spectrometer (MERIS), the Advanced Along-Track Scanning Radiometer (AATSR), and their combination. The cloud detection schemes were designed to be numerically efficient and suited for the processing of large numbers of data. Results from the classical and naive approach to Bayesian cloud masking are discussed for MERIS and AATSR as well as for their combination. A sensitivity study on the resolution of multidimensional histograms, which were post-processed by Gaussian smoothing, shows how theoretically insufficient numbers of truth data can be used to set up accurate classical Bayesian cloud masks. Sets of exploited features from single and derived channels are numerically optimized and results for naive and classical Bayesian cloud masks are presented. The application of the Bayesian approach is discussed in terms of reproducing existing algorithms, enhancing existing algorithms, increasing the robustness of existing algorithms, and on setting up new classification schemes based on manually classified scenes.

Modeling and parameterization of horizontally inhomogeneous cloud radiative properties

NASA Technical Reports Server (NTRS)

Welch, R. M.

1995-01-01

One of the fundamental difficulties in modeling cloud fields is the large variability of cloud optical properties (liquid water content, reflectance, emissivity). The stratocumulus and cirrus clouds, under special consideration for FIRE, exhibit spatial variability on scales of 1 km or less. While it is impractical to model individual cloud elements, the research direction is to model a statistical ensembles of cloud elements with mean-cloud properties specified. The major areas of this investigation are: (1) analysis of cloud field properties; (2) intercomparison of cloud radiative model results with satellite observations; (3) radiative parameterization of cloud fields; and (4) development of improved cloud classification algorithms.

Combining Passive Microwave Rain Rate Retrieval with Visible and Infrared Cloud Classification.

NASA Astrophysics Data System (ADS)

Miller, Shawn William

The relation between cloud type and rain rate has been investigated here from different approaches. Previous studies and intercomparisons have indicated that no single passive microwave rain rate algorithm is an optimal choice for all types of precipitating systems. Motivated by the upcoming Tropical Rainfall Measuring Mission (TRMM), an algorithm which combines visible and infrared cloud classification with passive microwave rain rate estimation was developed and analyzed in a preliminary manner using data from the Tropical Ocean Global Atmosphere-Coupled Ocean Atmosphere Response Experiment (TOGA-COARE). Overall correlation with radar rain rate measurements across five case studies showed substantial improvement in the combined algorithm approach when compared to the use of any single microwave algorithm. An automated neural network cloud classifier for use over both land and ocean was independently developed and tested on Advanced Very High Resolution Radiometer (AVHRR) data. The global classifier achieved strict accuracy for 82% of the test samples, while a more localized version achieved strict accuracy for 89% of its own test set. These numbers provide hope for the eventual development of a global automated cloud classifier for use throughout the tropics and the temperate zones. The localized classifier was used in conjunction with gridded 15-minute averaged radar rain rates at 8km resolution produced from the current operational network of National Weather Service (NWS) radars, to investigate the relation between cloud type and rain rate over three regions of the continental United States and adjacent waters. The results indicate a substantially lower amount of available moisture in the Front Range of the Rocky Mountains than in the Midwest or in the eastern Gulf of Mexico.

Multi-sensor measurements of mixed-phase clouds above Greenland

NASA Astrophysics Data System (ADS)

Stillwell, Robert A.; Shupe, Matthew D.; Thayer, Jeffrey P.; Neely, Ryan R.; Turner, David D.

2018-04-01

Liquid-only and mixed-phase clouds in the Arctic strongly affect the regional surface energy and ice mass budgets, yet much remains unknown about the nature of these clouds due to the lack of intensive measurements. Lidar measurements of these clouds are challenged by very large signal dynamic range, which makes even seemingly simple tasks, such as thermodynamic phase classification, difficult. This work focuses on a set of measurements made by the Clouds Aerosol Polarization and Backscatter Lidar at Summit, Greenland and its retrieval algorithms, which use both analog and photon counting as well as orthogonal and non-orthogonal polarization retrievals to extend dynamic range and improve overall measurement quality and quantity. Presented here is an algorithm for cloud parameter retrievals that leverages enhanced dynamic range retrievals to classify mixed-phase clouds. This best guess retrieval is compared to co-located instruments for validation.

Automatic Detection and Classification of Pole-Like Objects for Urban Cartography Using Mobile Laser Scanning Data

PubMed Central

Ordóñez, Celestino; Cabo, Carlos; Sanz-Ablanedo, Enoc

2017-01-01

Mobile laser scanning (MLS) is a modern and powerful technology capable of obtaining massive point clouds of objects in a short period of time. Although this technology is nowadays being widely applied in urban cartography and 3D city modelling, it has some drawbacks that need to be avoided in order to strengthen it. One of the most important shortcomings of MLS data is concerned with the fact that it provides an unstructured dataset whose processing is very time-consuming. Consequently, there is a growing interest in developing algorithms for the automatic extraction of useful information from MLS point clouds. This work is focused on establishing a methodology and developing an algorithm to detect pole-like objects and classify them into several categories using MLS datasets. The developed procedure starts with the discretization of the point cloud by means of a voxelization, in order to simplify and reduce the processing time in the segmentation process. In turn, a heuristic segmentation algorithm was developed to detect pole-like objects in the MLS point cloud. Finally, two supervised classification algorithms, linear discriminant analysis and support vector machines, were used to distinguish between the different types of poles in the point cloud. The predictors are the principal component eigenvalues obtained from the Cartesian coordinates of the laser points, the range of the Z coordinate, and some shape-related indexes. The performance of the method was tested in an urban area with 123 poles of different categories. Very encouraging results were obtained, since the accuracy rate was over 90%. PMID:28640189

Characterizing relative humidity with respect to ice in midlatitude cirrus clouds as a function of atmospheric state

NASA Astrophysics Data System (ADS)

Dzambo, Andrew M.; Turner, David D.

2016-10-01

Midlatitude cirrus cloud macrophysical and microphysical properties have been shown in previous studies to vary seasonally and in various large-scale dynamical regimes, but relative humidity with respect to ice (RHI) within cirrus clouds has not been studied extensively in this context. Using a combination of radiosonde and millimeter-wavelength cloud radar data, we identify 1076 cirrus clouds spanning a 7 year period from 2004 to 2011. These data are separated into five classes using a previously published algorithm that is based largely on synoptic conditions. Using these data and classification scheme, we find that RHI in cirrus clouds varies seasonally. Variations in cirrus cloud RHI exist within the prescribed classifications; however, most of the variations are within the measurement uncertainty. Additionally, with the exception of nonsummer class cirrus, these variations are not statistically significant. We also find that cirrus cloud occurrence is not necessarily correlated with higher observed values of RHI. The structure of RHI in cirrus clouds varies more in thicker clouds, which follows previous studies showing that macrophysical and microphysical variability increases in thicker cirrus clouds.

The effects of cloud inhomogeneities upon radiative fluxes, and the supply of a cloud truth validation dataset

NASA Technical Reports Server (NTRS)

Welch, Ronald M.

1993-01-01

A series of cloud and sea ice retrieval algorithms are being developed in support of the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) Science Team objectives. These retrievals include the following: cloud fractional area, cloud optical thickness, cloud phase (water or ice), cloud particle effective radius, cloud top heights, cloud base height, cloud top temperature, cloud emissivity, cloud 3-D structure, cloud field scales of organization, sea ice fractional area, sea ice temperature, sea ice albedo, and sea surface temperature. Due to the problems of accurately retrieving cloud properties over bright surfaces, an advanced cloud classification method was developed which is based upon spectral and textural features and artificial intelligence classifiers.

A neural network approach to cloud classification

NASA Technical Reports Server (NTRS)

Lee, Jonathan; Weger, Ronald C.; Sengupta, Sailes K.; Welch, Ronald M.

1990-01-01

It is shown that, using high-spatial-resolution data, very high cloud classification accuracies can be obtained with a neural network approach. A texture-based neural network classifier using only single-channel visible Landsat MSS imagery achieves an overall cloud identification accuracy of 93 percent. Cirrus can be distinguished from boundary layer cloudiness with an accuracy of 96 percent, without the use of an infrared channel. Stratocumulus is retrieved with an accuracy of 92 percent, cumulus at 90 percent. The use of the neural network does not improve cirrus classification accuracy. Rather, its main effect is in the improved separation between stratocumulus and cumulus cloudiness. While most cloud classification algorithms rely on linear parametric schemes, the present study is based on a nonlinear, nonparametric four-layer neural network approach. A three-layer neural network architecture, the nonparametric K-nearest neighbor approach, and the linear stepwise discriminant analysis procedure are compared. A significant finding is that significantly higher accuracies are attained with the nonparametric approaches using only 20 percent of the database as training data, compared to 67 percent of the database in the linear approach.

An Automatic Cloud Mask Algorithm Based on Time Series of MODIS Measurements

NASA Technical Reports Server (NTRS)

Lyapustin, Alexei; Wang, Yujie; Frey, R.

2008-01-01

Quality of aerosol retrievals and atmospheric correction depends strongly on accuracy of the cloud mask (CM) algorithm. The heritage CM algorithms developed for AVHRR and MODIS use the latest sensor measurements of spectral reflectance and brightness temperature and perform processing at the pixel level. The algorithms are threshold-based and empirically tuned. They don't explicitly address the classical problem of cloud search, wherein the baseline clear-skies scene is defined for comparison. Here, we report on a new CM algorithm which explicitly builds and maintains a reference clear-skies image of the surface (refcm) using a time series of MODIS measurements. The new algorithm, developed as part of the Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm for MODIS, relies on fact that clear-skies images of the same surface area have a common textural pattern, defined by the surface topography, boundaries of rivers and lakes, distribution of soils and vegetation etc. This pattern changes slowly given the daily rate of global Earth observations, whereas clouds introduce high-frequency random disturbances. Under clear skies, consecutive gridded images of the same surface area have a high covariance, whereas in presence of clouds covariance is usually low. This idea is central to initialization of refcm which is used to derive cloud mask in combination with spectral and brightness temperature tests. The refcm is continuously updated with the latest clear-skies MODIS measurements, thus adapting to seasonal and rapid surface changes. The algorithm is enhanced by an internal dynamic land-water-snow classification coupled with a surface change mask. An initial comparison shows that the new algorithm offers the potential to perform better than the MODIS MOD35 cloud mask in situations where the land surface is changing rapidly, and over Earth regions covered by snow and ice.

Classifying Structures in the ISM with Machine Learning Techniques

NASA Astrophysics Data System (ADS)

Beaumont, Christopher; Goodman, A. A.; Williams, J. P.

2011-01-01

The processes which govern molecular cloud evolution and star formation often sculpt structures in the ISM: filaments, pillars, shells, outflows, etc. Because of their morphological complexity, these objects are often identified manually. Manual classification has several disadvantages; the process is subjective, not easily reproducible, and does not scale well to handle increasingly large datasets. We have explored to what extent machine learning algorithms can be trained to autonomously identify specific morphological features in molecular cloud datasets. We show that the Support Vector Machine algorithm can successfully locate filaments and outflows blended with other emission structures. When the objects of interest are morphologically distinct from the surrounding emission, this autonomous classification achieves >90% accuracy. We have developed a set of IDL-based tools to apply this technique to other datasets.

Automatic Classification of Trees from Laser Scanning Point Clouds

NASA Astrophysics Data System (ADS)

Sirmacek, B.; Lindenbergh, R.

2015-08-01

Development of laser scanning technologies has promoted tree monitoring studies to a new level, as the laser scanning point clouds enable accurate 3D measurements in a fast and environmental friendly manner. In this paper, we introduce a probability matrix computation based algorithm for automatically classifying laser scanning point clouds into 'tree' and 'non-tree' classes. Our method uses the 3D coordinates of the laser scanning points as input and generates a new point cloud which holds a label for each point indicating if it belongs to the 'tree' or 'non-tree' class. To do so, a grid surface is assigned to the lowest height level of the point cloud. The grids are filled with probability values which are calculated by checking the point density above the grid. Since the tree trunk locations appear with very high values in the probability matrix, selecting the local maxima of the grid surface help to detect the tree trunks. Further points are assigned to tree trunks if they appear in the close proximity of trunks. Since heavy mathematical computations (such as point cloud organization, detailed shape 3D detection methods, graph network generation) are not required, the proposed algorithm works very fast compared to the existing methods. The tree classification results are found reliable even on point clouds of cities containing many different objects. As the most significant weakness, false detection of light poles, traffic signs and other objects close to trees cannot be prevented. Nevertheless, the experimental results on mobile and airborne laser scanning point clouds indicate the possible usage of the algorithm as an important step for tree growth observation, tree counting and similar applications. While the laser scanning point cloud is giving opportunity to classify even very small trees, accuracy of the results is reduced in the low point density areas further away than the scanning location. These advantages and disadvantages of two laser scanning point cloud sources are discussed in detail.

Person detection and tracking with a 360° lidar system

NASA Astrophysics Data System (ADS)

Hammer, Marcus; Hebel, Marcus; Arens, Michael

2017-10-01

Today it is easily possible to generate dense point clouds of the sensor environment using 360° LiDAR (Light Detection and Ranging) sensors which are available since a number of years. The interpretation of these data is much more challenging. For the automated data evaluation the detection and classification of objects is a fundamental task. Especially in urban scenarios moving objects like persons or vehicles are of particular interest, for instance in automatic collision avoidance, for mobile sensor platforms or surveillance tasks. In literature there are several approaches for automated person detection in point clouds. While most techniques show acceptable results in object detection, the computation time is often crucial. The runtime can be problematic, especially due to the amount of data in the panoramic 360° point clouds. On the other hand, for most applications an object detection and classification in real time is needed. The paper presents a proposal for a fast, real-time capable algorithm for person detection, classification and tracking in panoramic point clouds.

A Practical and Automated Approach to Large Area Forest Disturbance Mapping with Remote Sensing

PubMed Central

Ozdogan, Mutlu

2014-01-01

In this paper, I describe a set of procedures that automate forest disturbance mapping using a pair of Landsat images. The approach is built on the traditional pair-wise change detection method, but is designed to extract training data without user interaction and uses a robust classification algorithm capable of handling incorrectly labeled training data. The steps in this procedure include: i) creating masks for water, non-forested areas, clouds, and cloud shadows; ii) identifying training pixels whose value is above or below a threshold defined by the number of standard deviations from the mean value of the histograms generated from local windows in the short-wave infrared (SWIR) difference image; iii) filtering the original training data through a number of classification algorithms using an n-fold cross validation to eliminate mislabeled training samples; and finally, iv) mapping forest disturbance using a supervised classification algorithm. When applied to 17 Landsat footprints across the U.S. at five-year intervals between 1985 and 2010, the proposed approach produced forest disturbance maps with 80 to 95% overall accuracy, comparable to those obtained from traditional approaches to forest change detection. The primary sources of mis-classification errors included inaccurate identification of forests (errors of commission), issues related to the land/water mask, and clouds and cloud shadows missed during image screening. The approach requires images from the peak growing season, at least for the deciduous forest sites, and cannot readily distinguish forest harvest from natural disturbances or other types of land cover change. The accuracy of detecting forest disturbance diminishes with the number of years between the images that make up the image pair. Nevertheless, the relatively high accuracies, little or no user input needed for processing, speed of map production, and simplicity of the approach make the new method especially practical for forest cover change analysis over very large regions. PMID:24717283

A practical and automated approach to large area forest disturbance mapping with remote sensing.

PubMed

Ozdogan, Mutlu

2014-01-01

In this paper, I describe a set of procedures that automate forest disturbance mapping using a pair of Landsat images. The approach is built on the traditional pair-wise change detection method, but is designed to extract training data without user interaction and uses a robust classification algorithm capable of handling incorrectly labeled training data. The steps in this procedure include: i) creating masks for water, non-forested areas, clouds, and cloud shadows; ii) identifying training pixels whose value is above or below a threshold defined by the number of standard deviations from the mean value of the histograms generated from local windows in the short-wave infrared (SWIR) difference image; iii) filtering the original training data through a number of classification algorithms using an n-fold cross validation to eliminate mislabeled training samples; and finally, iv) mapping forest disturbance using a supervised classification algorithm. When applied to 17 Landsat footprints across the U.S. at five-year intervals between 1985 and 2010, the proposed approach produced forest disturbance maps with 80 to 95% overall accuracy, comparable to those obtained from traditional approaches to forest change detection. The primary sources of mis-classification errors included inaccurate identification of forests (errors of commission), issues related to the land/water mask, and clouds and cloud shadows missed during image screening. The approach requires images from the peak growing season, at least for the deciduous forest sites, and cannot readily distinguish forest harvest from natural disturbances or other types of land cover change. The accuracy of detecting forest disturbance diminishes with the number of years between the images that make up the image pair. Nevertheless, the relatively high accuracies, little or no user input needed for processing, speed of map production, and simplicity of the approach make the new method especially practical for forest cover change analysis over very large regions.

Cloud, Aerosol, and Volcanic Ash Retrievals Using ASTR and SLSTR with ORAC

NASA Astrophysics Data System (ADS)

McGarragh, Gregory; Poulsen, Caroline; Povey, Adam; Thomas, Gareth; Christensen, Matt; Sus, Oliver; Schlundt, Cornelia; Stapelberg, Stefan; Stengel, Martin; Grainger, Don

2015-12-01

The Optimal Retrieval of Aerosol and Cloud (ORAC) is a generalized optimal estimation system that retrieves cloud, aerosol and volcanic ash parameters using satellite imager measurements in the visible to infrared. Use of the same algorithm for different sensors and parameters leads to consistency that facilitates inter-comparison and interaction studies. ORAC currently supports ATSR, AVHRR, MODIS and SEVIRI. In this proceeding we discuss the ORAC retrieval algorithm applied to ATSR data including the retrieval methodology, the forward model, uncertainty characterization and discrimination/classification techniques. Application of ORAC to SLSTR data is discussed including the additional features that SLSTR provides relative to the ATSR heritage. The ORAC level 2 and level 3 results are discussed and an application of level 3 results to the study of cloud/aerosol interactions is presented.

Building Change Detection from Bi-Temporal Dense-Matching Point Clouds and Aerial Images.

PubMed

Pang, Shiyan; Hu, Xiangyun; Cai, Zhongliang; Gong, Jinqi; Zhang, Mi

2018-03-24

In this work, a novel building change detection method from bi-temporal dense-matching point clouds and aerial images is proposed to address two major problems, namely, the robust acquisition of the changed objects above ground and the automatic classification of changed objects into buildings or non-buildings. For the acquisition of changed objects above ground, the change detection problem is converted into a binary classification, in which the changed area above ground is regarded as the foreground and the other area as the background. For the gridded points of each period, the graph cuts algorithm is adopted to classify the points into foreground and background, followed by the region-growing algorithm to form candidate changed building objects. A novel structural feature that was extracted from aerial images is constructed to classify the candidate changed building objects into buildings and non-buildings. The changed building objects are further classified as "newly built", "taller", "demolished", and "lower" by combining the classification and the digital surface models of two periods. Finally, three typical areas from a large dataset are used to validate the proposed method. Numerous experiments demonstrate the effectiveness of the proposed algorithm.

Automated Visibility & Cloud Cover Measurements with a Solid State Imaging System

DTIC Science & Technology

1989-03-01

GL-TR-89-0061 SIO Ref. 89-7 MPL-U-26/89 AUTOMATED VISIBILITY & CLOUD COVER MEASUREMENTS WITH A SOLID-STATE IMAGING SYSTEM C) to N4 R. W. Johnson W. S...include Security Classification) Automated Visibility & Cloud Measurements With A Solid State Imaging System 12. PERSONAL AUTHOR(S) Richard W. Johnson...based imaging systems , their ics and control algorithms, thus they ar.L discussed sepa- initial deployment and the preliminary application of rately

Invariant-feature-based adaptive automatic target recognition in obscured 3D point clouds

NASA Astrophysics Data System (ADS)

Khuon, Timothy; Kershner, Charles; Mattei, Enrico; Alverio, Arnel; Rand, Robert

2014-06-01

Target recognition and classification in a 3D point cloud is a non-trivial process due to the nature of the data collected from a sensor system. The signal can be corrupted by noise from the environment, electronic system, A/D converter, etc. Therefore, an adaptive system with a desired tolerance is required to perform classification and recognition optimally. The feature-based pattern recognition algorithm architecture as described below is particularly devised for solving a single-sensor classification non-parametrically. Feature set is extracted from an input point cloud, normalized, and classifier a neural network classifier. For instance, automatic target recognition in an urban area would require different feature sets from one in a dense foliage area. The figure above (see manuscript) illustrates the architecture of the feature based adaptive signature extraction of 3D point cloud including LIDAR, RADAR, and electro-optical data. This network takes a 3D cluster and classifies it into a specific class. The algorithm is a supervised and adaptive classifier with two modes: the training mode and the performing mode. For the training mode, a number of novel patterns are selected from actual or artificial data. A particular 3D cluster is input to the network as shown above for the decision class output. The network consists of three sequential functional modules. The first module is for feature extraction that extracts the input cluster into a set of singular value features or feature vector. Then the feature vector is input into the feature normalization module to normalize and balance it before being fed to the neural net classifier for the classification. The neural net can be trained by actual or artificial novel data until each trained output reaches the declared output within the defined tolerance. In case new novel data is added after the neural net has been learned, the training is then resumed until the neural net has incrementally learned with the new novel data. The associative memory capability of the neural net enables the incremental learning. The back propagation algorithm or support vector machine can be utilized for the classification and recognition.

Large Scale Gaussian Processes for Atmospheric Parameter Retrieval and Cloud Screening

NASA Astrophysics Data System (ADS)

Camps-Valls, G.; Gomez-Chova, L.; Mateo, G.; Laparra, V.; Perez-Suay, A.; Munoz-Mari, J.

2017-12-01

Current Earth-observation (EO) applications for image classification have to deal with an unprecedented big amount of heterogeneous and complex data sources. Spatio-temporally explicit classification methods are a requirement in a variety of Earth system data processing applications. Upcoming missions such as the super-spectral Copernicus Sentinels EnMAP and FLEX will soon provide unprecedented data streams. Very high resolution (VHR) sensors like Worldview-3 also pose big challenges to data processing. The challenge is not only attached to optical sensors but also to infrared sounders and radar images which increased in spectral, spatial and temporal resolution. Besides, we should not forget the availability of the extremely large remote sensing data archives already collected by several past missions, such ENVISAT, Cosmo-SkyMED, Landsat, SPOT, or Seviri/MSG. These large-scale data problems require enhanced processing techniques that should be accurate, robust and fast. Standard parameter retrieval and classification algorithms cannot cope with this new scenario efficiently. In this work, we review the field of large scale kernel methods for both atmospheric parameter retrieval and cloud detection using infrared sounding IASI data and optical Seviri/MSG imagery. We propose novel Gaussian Processes (GPs) to train problems with millions of instances and high number of input features. Algorithms can cope with non-linearities efficiently, accommodate multi-output problems, and provide confidence intervals for the predictions. Several strategies to speed up algorithms are devised: random Fourier features and variational approaches for cloud classification using IASI data and Seviri/MSG, and engineered randomized kernel functions and emulation in temperature, moisture and ozone atmospheric profile retrieval from IASI as a proxy to the upcoming MTG-IRS sensor. Excellent compromise between accuracy and scalability are obtained in all applications.

Classification of LIDAR Data for Generating a High-Precision Roadway Map

NASA Astrophysics Data System (ADS)

Jeong, J.; Lee, I.

2016-06-01

Generating of a highly precise map grows up with development of autonomous driving vehicles. The highly precise map includes a precision of centimetres level unlike an existing commercial map with the precision of meters level. It is important to understand road environments and make a decision for autonomous driving since a robust localization is one of the critical challenges for the autonomous driving car. The one of source data is from a Lidar because it provides highly dense point cloud data with three dimensional position, intensities and ranges from the sensor to target. In this paper, we focus on how to segment point cloud data from a Lidar on a vehicle and classify objects on the road for the highly precise map. In particular, we propose the combination with a feature descriptor and a classification algorithm in machine learning. Objects can be distinguish by geometrical features based on a surface normal of each point. To achieve correct classification using limited point cloud data sets, a Support Vector Machine algorithm in machine learning are used. Final step is to evaluate accuracies of obtained results by comparing them to reference data The results show sufficient accuracy and it will be utilized to generate a highly precise road map.

Detection of Multi-Layer and Vertically-Extended Clouds Using A-Train Sensors

NASA Technical Reports Server (NTRS)

Joiner, J.; Vasilkov, A. P.; Bhartia, P. K.; Wind, G.; Platnick, S.; Menzel, W. P.

2010-01-01

The detection of mUltiple cloud layers using satellite observations is important for retrieval algorithms as well as climate applications. In this paper, we describe a relatively simple algorithm to detect multiple cloud layers and distinguish them from vertically-extended clouds. The algorithm can be applied to coincident passive sensors that derive both cloud-top pressure from the thermal infrared observations and an estimate of solar photon pathlength from UV, visible, or near-IR measurements. Here, we use data from the A-train afternoon constellation of satellites: cloud-top pressure, cloud optical thickness, the multi-layer flag from the Aqua MODerate-resolution Imaging Spectroradiometer (MODIS) and the optical centroid cloud pressure from the Aura Ozone Monitoring Instrument (OMI). For the first time, we use data from the CloudSat radar to evaluate the results of a multi-layer cloud detection scheme. The cloud classification algorithms applied with different passive sensor configurations compare well with each other as well as with data from CloudSat. We compute monthly mean fractions of pixels containing multi-layer and vertically-extended clouds for January and July 2007 at the OMI spatial resolution (l2kmx24km at nadir) and at the 5kmx5km MODIS resolution used for infrared cloud retrievals. There are seasonal variations in the spatial distribution of the different cloud types. The fraction of cloudy pixels containing distinct multi-layer cloud is a strong function of the pixel size. Globally averaged, these fractions are approximately 20% and 10% for OMI and MODIS, respectively. These fractions may be significantly higher or lower depending upon location. There is a much smaller resolution dependence for fractions of pixels containing vertically-extended clouds (approx.20% for OMI and slightly less for MODIS globally), suggesting larger spatial scales for these clouds. We also find higher fractions of vertically-extended clouds over land as compared with ocean, particularly in the tropics and summer hemisphere.

A path towards uncertainty assignment in an operational cloud-phase algorithm from ARM vertically pointing active sensors

DOE PAGES

Riihimaki, Laura D.; Comstock, Jennifer M.; Anderson, Kevin K.; ...

2016-06-10

Knowledge of cloud phase (liquid, ice, mixed, etc.) is necessary to describe the radiative impact of clouds and their lifetimes, but is a property that is difficult to simulate correctly in climate models. One step towards improving those simulations is to make observations of cloud phase with sufficient accuracy to help constrain model representations of cloud processes. In this study, we outline a methodology using a basic Bayesian classifier to estimate the probabilities of cloud-phase class from Atmospheric Radiation Measurement (ARM) vertically pointing active remote sensors. The advantage of this method over previous ones is that it provides uncertainty informationmore » on the phase classification. We also test the value of including higher moments of the cloud radar Doppler spectrum than are traditionally used operationally. Using training data of known phase from the Mixed-Phase Arctic Cloud Experiment (M-PACE) field campaign, we demonstrate a proof of concept for how the method can be used to train an algorithm that identifies ice, liquid, mixed phase, and snow. Over 95 % of data are identified correctly for pure ice and liquid cases used in this study. Mixed-phase and snow cases are more problematic to identify correctly. When lidar data are not available, including additional information from the Doppler spectrum provides substantial improvement to the algorithm. As a result, this is a first step towards an operational algorithm and can be expanded to include additional categories such as drizzle with additional training data.« less

A path towards uncertainty assignment in an operational cloud-phase algorithm from ARM vertically pointing active sensors

NASA Astrophysics Data System (ADS)

Riihimaki, Laura D.; Comstock, Jennifer M.; Anderson, Kevin K.; Holmes, Aimee; Luke, Edward

2016-06-01

Knowledge of cloud phase (liquid, ice, mixed, etc.) is necessary to describe the radiative impact of clouds and their lifetimes, but is a property that is difficult to simulate correctly in climate models. One step towards improving those simulations is to make observations of cloud phase with sufficient accuracy to help constrain model representations of cloud processes. In this study, we outline a methodology using a basic Bayesian classifier to estimate the probabilities of cloud-phase class from Atmospheric Radiation Measurement (ARM) vertically pointing active remote sensors. The advantage of this method over previous ones is that it provides uncertainty information on the phase classification. We also test the value of including higher moments of the cloud radar Doppler spectrum than are traditionally used operationally. Using training data of known phase from the Mixed-Phase Arctic Cloud Experiment (M-PACE) field campaign, we demonstrate a proof of concept for how the method can be used to train an algorithm that identifies ice, liquid, mixed phase, and snow. Over 95 % of data are identified correctly for pure ice and liquid cases used in this study. Mixed-phase and snow cases are more problematic to identify correctly. When lidar data are not available, including additional information from the Doppler spectrum provides substantial improvement to the algorithm. This is a first step towards an operational algorithm and can be expanded to include additional categories such as drizzle with additional training data.

Continuous Change Detection and Classification (CCDC) of Land Cover Using All Available Landsat Data

NASA Astrophysics Data System (ADS)

Zhu, Z.; Woodcock, C. E.

2012-12-01

A new algorithm for Continuous Change Detection and Classification (CCDC) of land cover using all available Landsat data is developed. This new algorithm is capable of detecting many kinds of land cover change as new images are collected and at the same time provide land cover maps for any given time. To better identify land cover change, a two step cloud, cloud shadow, and snow masking algorithm is used for eliminating "noisy" observations. Next, a time series model that has components of seasonality, trend, and break estimates the surface reflectance and temperature. The time series model is updated continuously with newly acquired observations. Due to the high variability in spectral response for different kinds of land cover change, the CCDC algorithm uses a data-driven threshold derived from all seven Landsat bands. When the difference between observed and predicted exceeds the thresholds three consecutive times, a pixel is identified as land cover change. Land cover classification is done after change detection. Coefficients from the time series models and the Root Mean Square Error (RMSE) from model fitting are used as classification inputs for the Random Forest Classifier (RFC). We applied this new algorithm for one Landsat scene (Path 12 Row 31) that includes all of Rhode Island as well as much of Eastern Massachusetts and parts of Connecticut. A total of 532 Landsat images acquired between 1982 and 2011 were processed. During this period, 619,924 pixels were detected to change once (91% of total changed pixels) and 60,199 pixels were detected to change twice (8% of total changed pixels). The most frequent land cover change category is from mixed forest to low density residential which occupies more than 8% of total land cover change pixels.

Investigation of cloud/water vapor motion winds from geostationary satellite

NASA Technical Reports Server (NTRS)

Nieman, Steve; Velden, Chris; Hayden, Kit; Menzel, Paul

1993-01-01

Work has been primarily focussed on three tasks: (1) comparison of wind fields produced at MSFC with the CO2 autowind/autoeditor system newly installed in NESDIS operations; (2) evaluation of techniques for improved tracer selection through use of cloud classification predictors; and (3) development of height assignment algorithm with water vapor channel radiances. The contract goal is to improve the CIMSS wind system by developing new techniques and assimilating better existing techniques. The work reported here was done in collaboration with the NESDIS scientists working on the operational winds software, so that NASA funded research can benefit NESDIS operational algorithms.

Optically Thin Liquid Water Clouds: Their Importance and Our Challenge

NASA Technical Reports Server (NTRS)

Turner, D. D.; Vogelmann, A. M.; Austin, R. T.; Barnard, J. C.; Cady-Pereira, K.; Chiu, J. C.; Clough, S. A.; Flynn, C.; Khaiyer, M. M.; Liljegren, J.;

An approach for combining airborne LiDAR and high-resolution aerial color imagery using Gaussian processes

NASA Astrophysics Data System (ADS)

Liu, Yansong; Monteiro, Sildomar T.; Saber, Eli

2015-10-01

Changes in vegetation cover, building construction, road network and traffic conditions caused by urban expansion affect the human habitat as well as the natural environment in rapidly developing cities. It is crucial to assess these changes and respond accordingly by identifying man-made and natural structures with accurate classification algorithms. With the increase in use of multi-sensor remote sensing systems, researchers are able to obtain a more complete description of the scene of interest. By utilizing multi-sensor data, the accuracy of classification algorithms can be improved. In this paper, we propose a method for combining 3D LiDAR point clouds and high-resolution color images to classify urban areas using Gaussian processes (GP). GP classification is a powerful non-parametric classification method that yields probabilistic classification results. It makes predictions in a way that addresses the uncertainty of real world. In this paper, we attempt to identify man-made and natural objects in urban areas including buildings, roads, trees, grass, water and vehicles. LiDAR features are derived from the 3D point clouds and the spatial and color features are extracted from RGB images. For classification, we use the Laplacian approximation for GP binary classification on the new combined feature space. The multiclass classification has been implemented by using one-vs-all binary classification strategy. The result of applying support vector machines (SVMs) and logistic regression (LR) classifier is also provided for comparison. Our experiments show a clear improvement of classification results by using the two sensors combined instead of each sensor separately. Also we found the advantage of applying GP approach to handle the uncertainty in classification result without compromising accuracy compared to SVM, which is considered as the state-of-the-art classification method.

a Point Cloud Classification Approach Based on Vertical Structures of Ground Objects

NASA Astrophysics Data System (ADS)

Zhao, Y.; Hu, Q.; Hu, W.

2018-04-01

This paper proposes a novel method for point cloud classification using vertical structural characteristics of ground objects. Since urbanization develops rapidly nowadays, urban ground objects also change frequently. Conventional photogrammetric methods cannot satisfy the requirements of updating the ground objects' information efficiently, so LiDAR (Light Detection and Ranging) technology is employed to accomplish this task. LiDAR data, namely point cloud data, can obtain detailed three-dimensional coordinates of ground objects, but this kind of data is discrete and unorganized. To accomplish ground objects classification with point cloud, we first construct horizontal grids and vertical layers to organize point cloud data, and then calculate vertical characteristics, including density and measures of dispersion, and form characteristic curves for each grids. With the help of PCA processing and K-means algorithm, we analyze the similarities and differences of characteristic curves. Curves that have similar features will be classified into the same class and point cloud correspond to these curves will be classified as well. The whole process is simple but effective, and this approach does not need assistance of other data sources. In this study, point cloud data are classified into three classes, which are vegetation, buildings, and roads. When horizontal grid spacing and vertical layer spacing are 3 m and 1 m respectively, vertical characteristic is set as density, and the number of dimensions after PCA processing is 11, the overall precision of classification result is about 86.31 %. The result can help us quickly understand the distribution of various ground objects.

Cloud and aerosol studies using combined CPL and MAS data

NASA Astrophysics Data System (ADS)

Vaughan, Mark A.; Rodier, Sharon; Hu, Yongxiang; McGill, Matthew J.; Holz, Robert E.

2004-11-01

Current uncertainties in the role of aerosols and clouds in the Earth's climate system limit our abilities to model the climate system and predict climate change. These limitations are due primarily to difficulties of adequately measuring aerosols and clouds on a global scale. The A-train satellites (Aqua, CALIPSO, CloudSat, PARASOL, and Aura) will provide an unprecedented opportunity to address these uncertainties. The various active and passive sensors of the A-train will use a variety of measurement techniques to provide comprehensive observations of the multi-dimensional properties of clouds and aerosols. However, to fully achieve the potential of this ensemble requires a robust data analysis framework to optimally and efficiently map these individual measurements into a comprehensive set of cloud and aerosol physical properties. In this work we introduce the Multi-Instrument Data Analysis and Synthesis (MIDAS) project, whose goal is to develop a suite of physically sound and computationally efficient algorithms that will combine active and passive remote sensing data in order to produce improved assessments of aerosol and cloud radiative and microphysical properties. These algorithms include (a) the development of an intelligent feature detection algorithm that combines inputs from both active and passive sensors, and (b) identifying recognizable multi-instrument signatures related to aerosol and cloud type derived from clusters of image pixels and the associated vertical profile information. Classification of these signatures will lead to the automated identification of aerosol and cloud types. Testing of these new algorithms is done using currently existing and readily available active and passive measurements from the Cloud Physics Lidar and the MODIS Airborne Simulator, which simulate, respectively, the CALIPSO and MODIS A-train instruments.

Orbiting Carbon Observatory-2 (OCO-2) Cloud Screening; Validation Against Collocated MODIS and Initial Comparison to CALIOP Data

NASA Technical Reports Server (NTRS)

Taylor, Thomas E.; O'Dell, Christopher W.; Frankenberg, Christian; Partain, Philip; Cronk, Heather W.; Savtchenko, Andrey; Nelson, Robert R.; Rosenthal, Emily J.; Chang, Albert; Crisp, David;

A New Cloud and Aerosol Layer Detection Method Based on Micropulse Lidar Measurements

NASA Astrophysics Data System (ADS)

Wang, Q.; Zhao, C.; Wang, Y.; Li, Z.; Wang, Z.; Liu, D.

2014-12-01

A new algorithm is developed to detect aerosols and clouds based on micropulse lidar (MPL) measurements. In this method, a semi-discretization processing (SDP) technique is first used to inhibit the impact of increasing noise with distance, then a value distribution equalization (VDE) method is introduced to reduce the magnitude of signal variations with distance. Combined with empirical threshold values, clouds and aerosols are detected and separated. This method can detect clouds and aerosols with high accuracy, although classification of aerosols and clouds is sensitive to the thresholds selected. Compared with the existing Atmospheric Radiation Measurement (ARM) program lidar-based cloud product, the new method detects more high clouds. The algorithm was applied to a year of observations at both the U.S. Southern Great Plains (SGP) and China Taihu site. At SGP, the cloud frequency shows a clear seasonal variation with maximum values in winter and spring, and shows bi-modal vertical distributions with maximum frequency at around 3-6 km and 8-12 km. The annual averaged cloud frequency is about 50%. By contrast, the cloud frequency at Taihu shows no clear seasonal variation and the maximum frequency is at around 1 km. The annual averaged cloud frequency is about 15% higher than that at SGP.

Aerosol properties, source identification, and cloud processing in orographic clouds measured by single particle mass spectrometry on a central European mountain site during HCCT-2010

NASA Astrophysics Data System (ADS)

Roth, A.; Schneider, J.; Klimach, T.; Mertes, S.; van Pinxteren, D.; Herrmann, H.; Borrmann, S.

2016-01-01

Cloud residues and out-of-cloud aerosol particles with diameters between 150 and 900 nm were analysed by online single particle aerosol mass spectrometry during the 6-week study Hill Cap Cloud Thuringia (HCCT)-2010 in September-October 2010. The measurement location was the mountain Schmücke (937 m a.s.l.) in central Germany. More than 160 000 bipolar mass spectra from out-of-cloud aerosol particles and more than 13 000 bipolar mass spectra from cloud residual particles were obtained and were classified using a fuzzy c-means clustering algorithm. Analysis of the uncertainty of the sorting algorithm was conducted on a subset of the data by comparing the clustering output with particle-by-particle inspection and classification by the operator. This analysis yielded a false classification probability between 13 and 48 %. Additionally, particle types were identified by specific marker ions. The results from the ambient aerosol analysis show that 63 % of the analysed particles belong to clusters having a diurnal variation, suggesting that local or regional sources dominate the aerosol, especially for particles containing soot and biomass burning particles. In the cloud residues, the relative percentage of large soot-containing particles and particles containing amines was found to be increased compared to the out-of-cloud aerosol, while, in general, organic particles were less abundant in the cloud residues. In the case of amines, this can be explained by the high solubility of the amines, while the large soot-containing particles were found to be internally mixed with inorganics, which explains their activation as cloud condensation nuclei. Furthermore, the results show that during cloud processing, both sulfate and nitrate are added to the residual particles, thereby changing the mixing state and increasing the fraction of particles with nitrate and/or sulfate. This is expected to lead to higher hygroscopicity after cloud evaporation, and therefore to an increase of the particles' ability to act as cloud condensation nuclei after their cloud passage.

Evolution in Cloud Population Statistics of the MJO: From AMIE Field Observations to Global-Cloud Permitting Models Final Report

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kollias, Pavlos

This is a multi-institutional, collaborative project using a three-tier modeling approach to bridge field observations and global cloud-permitting models, with emphases on cloud population structural evolution through various large-scale environments. Our contribution was in data analysis for the generation of high value cloud and precipitation products and derive cloud statistics for model validation. There are two areas in data analysis that we contributed: the development of a synergistic cloud and precipitation cloud classification that identify different cloud (e.g. shallow cumulus, cirrus) and precipitation types (shallow, deep, convective, stratiform) using profiling ARM observations and the development of a quantitative precipitation ratemore » retrieval algorithm using profiling ARM observations. Similar efforts have been developed in the past for precipitation (weather radars), but not for the millimeter-wavelength (cloud) radar deployed at the ARM sites.« less

Automated Classification of Heritage Buildings for As-Built Bim Using Machine Learning Techniques

NASA Astrophysics Data System (ADS)

Bassier, M.; Vergauwen, M.; Van Genechten, B.

2017-08-01

Semantically rich three dimensional models such as Building Information Models (BIMs) are increasingly used in digital heritage. They provide the required information to varying stakeholders during the different stages of the historic buildings life cyle which is crucial in the conservation process. The creation of as-built BIM models is based on point cloud data. However, manually interpreting this data is labour intensive and often leads to misinterpretations. By automatically classifying the point cloud, the information can be proccesed more effeciently. A key aspect in this automated scan-to-BIM process is the classification of building objects. In this research we look to automatically recognise elements in existing buildings to create compact semantic information models. Our algorithm efficiently extracts the main structural components such as floors, ceilings, roofs, walls and beams despite the presence of significant clutter and occlusions. More specifically, Support Vector Machines (SVM) are proposed for the classification. The algorithm is evaluated using real data of a variety of existing buildings. The results prove that the used classifier recognizes the objects with both high precision and recall. As a result, entire data sets are reliably labelled at once. The approach enables experts to better document and process heritage assets.

Study of sensor spectral responses and data processing algorithms and architectures for onboard feature identification

NASA Technical Reports Server (NTRS)

Huck, F. O.; Davis, R. E.; Fales, C. L.; Aherron, R. M.

1982-01-01

A computational model of the deterministic and stochastic processes involved in remote sensing is used to study spectral feature identification techniques for real-time onboard processing of data acquired with advanced earth-resources sensors. Preliminary results indicate that: Narrow spectral responses are advantageous; signal normalization improves mean-square distance (MSD) classification accuracy but tends to degrade maximum-likelihood (MLH) classification accuracy; and MSD classification of normalized signals performs better than the computationally more complex MLH classification when imaging conditions change appreciably from those conditions during which reference data were acquired. The results also indicate that autonomous categorization of TM signals into vegetation, bare land, water, snow and clouds can be accomplished with adequate reliability for many applications over a reasonably wide range of imaging conditions. However, further analysis is required to develop computationally efficient boundary approximation algorithms for such categorization.

Determine precipitation rates from visible and infrared satellite images of clouds by pattern recognition technique. Progress Report, 1 Jul. 1985 - 31 Mar. 1987 Ph.D. Thesis

NASA Technical Reports Server (NTRS)

Weinman, James A.; Garan, Louis

1987-01-01

A more advanced cloud pattern analysis algorithm was subsequently developed to take the shape and brightness of the various clouds into account in a manner that is more consistent with the human analyst's perception of GOES cloud imagery. The results of that classification scheme were compared with precipitation probabilities observed from ships of opportunity off the U.S. east coast to derive empirical regressions between cloud types and precipitation probability. The cloud morphology was then quantitatively and objectively used to map precipitation probabilities during two winter months during which severe cold air outbreaks were observed over the northwest Atlantic. Precipitation probabilities associated with various cloud types are summarized. Maps of precipitation probability derived from the cloud morphology analysis program for two months and the precipitation probability derived from thirty years of ship observation were observed.

Examining the NZESM Cloud representation with Self Organizing Maps

NASA Astrophysics Data System (ADS)

Schuddeboom, Alex; McDonald, Adrian; Parsons, Simon; Morgenstern, Olaf; Harvey, Mike

2017-04-01

Several different cloud regimes are identified from MODIS satellite data and the representation of these regimes within the New Zealand Earth System Model (NZESM) is examined. For the development of our cloud classification we utilize a neural network algorithm known as self organizing maps (SOMs) on MODIS cloud top pressure - cloud optical thickness joint histograms. To evaluate the representation of the cloud within NZESM, the frequency and geographical distribution of the regimes is compared between the NZESM and satellite data. This approach has the advantage of not only identifying differences, but also potentially giving additional information about the discrepancy such as in which regions or phases of cloud the differences are most prominent. To allow for a more direct comparison between datasets, the COSP satellite simulation software is applied to NZESM output. COSP works by simulating the observational processes linked to a satellite, within the GCM, so that data can be generated in a way that shares the particular observational bias of specific satellites. By taking the COSP joint histograms and comparing them to our existing classifications we can easily search for discrepancies between the observational data and the simulations without having to be cautious of biases introduced by the satellite. Preliminary results, based on data for 2008, show a significant decrease in overall cloud fraction in the NZESM compared to the MODIS satellite data. To better understand the nature of this discrepancy, the cloud fraction related to different cloud heights and phases were also analysed.

Change detection using landsat time series: A review of frequencies, preprocessing, algorithms, and applications

NASA Astrophysics Data System (ADS)

Zhu, Zhe

2017-08-01

The free and open access to all archived Landsat images in 2008 has completely changed the way of using Landsat data. Many novel change detection algorithms based on Landsat time series have been developed We present a comprehensive review of four important aspects of change detection studies based on Landsat time series, including frequencies, preprocessing, algorithms, and applications. We observed the trend that the more recent the study, the higher the frequency of Landsat time series used. We reviewed a series of image preprocessing steps, including atmospheric correction, cloud and cloud shadow detection, and composite/fusion/metrics techniques. We divided all change detection algorithms into six categories, including thresholding, differencing, segmentation, trajectory classification, statistical boundary, and regression. Within each category, six major characteristics of different algorithms, such as frequency, change index, univariate/multivariate, online/offline, abrupt/gradual change, and sub-pixel/pixel/spatial were analyzed. Moreover, some of the widely-used change detection algorithms were also discussed. Finally, we reviewed different change detection applications by dividing these applications into two categories, change target and change agent detection.

Introducing two Random Forest based methods for cloud detection in remote sensing images

NASA Astrophysics Data System (ADS)

Ghasemian, Nafiseh; Akhoondzadeh, Mehdi

2018-07-01

Cloud detection is a necessary phase in satellite images processing to retrieve the atmospheric and lithospheric parameters. Currently, some cloud detection methods based on Random Forest (RF) model have been proposed but they do not consider both spectral and textural characteristics of the image. Furthermore, they have not been tested in the presence of snow/ice. In this paper, we introduce two RF based algorithms, Feature Level Fusion Random Forest (FLFRF) and Decision Level Fusion Random Forest (DLFRF) to incorporate visible, infrared (IR) and thermal spectral and textural features (FLFRF) including Gray Level Co-occurrence Matrix (GLCM) and Robust Extended Local Binary Pattern (RELBP_CI) or visible, IR and thermal classifiers (DLFRF) for highly accurate cloud detection on remote sensing images. FLFRF first fuses visible, IR and thermal features. Thereafter, it uses the RF model to classify pixels to cloud, snow/ice and background or thick cloud, thin cloud and background. DLFRF considers visible, IR and thermal features (both spectral and textural) separately and inserts each set of features to RF model. Then, it holds vote matrix of each run of the model. Finally, it fuses the classifiers using the majority vote method. To demonstrate the effectiveness of the proposed algorithms, 10 Terra MODIS and 15 Landsat 8 OLI/TIRS images with different spatial resolutions are used in this paper. Quantitative analyses are based on manually selected ground truth data. Results show that after adding RELBP_CI to input feature set cloud detection accuracy improves. Also, the average cloud kappa values of FLFRF and DLFRF on MODIS images (1 and 0.99) are higher than other machine learning methods, Linear Discriminate Analysis (LDA), Classification And Regression Tree (CART), K Nearest Neighbor (KNN) and Support Vector Machine (SVM) (0.96). The average snow/ice kappa values of FLFRF and DLFRF on MODIS images (1 and 0.85) are higher than other traditional methods. The quantitative values on Landsat 8 images show similar trend. Consequently, while SVM and K-nearest neighbor show overestimation in predicting cloud and snow/ice pixels, our Random Forest (RF) based models can achieve higher cloud, snow/ice kappa values on MODIS and thin cloud, thick cloud and snow/ice kappa values on Landsat 8 images. Our algorithms predict both thin and thick cloud on Landsat 8 images while the existing cloud detection algorithm, Fmask cannot discriminate them. Compared to the state-of-the-art methods, our algorithms have acquired higher average cloud and snow/ice kappa values for different spatial resolutions.

Identification of stable areas in unreferenced laser scans for automated geomorphometric monitoring

NASA Astrophysics Data System (ADS)

Wujanz, Daniel; Avian, Michael; Krueger, Daniel; Neitzel, Frank

2018-04-01

Current research questions in the field of geomorphology focus on the impact of climate change on several processes subsequently causing natural hazards. Geodetic deformation measurements are a suitable tool to document such geomorphic mechanisms, e.g. by capturing a region of interest with terrestrial laser scanners which results in a so-called 3-D point cloud. The main problem in deformation monitoring is the transformation of 3-D point clouds captured at different points in time (epochs) into a stable reference coordinate system. In this contribution, a surface-based registration methodology is applied, termed the iterative closest proximity algorithm (ICProx), that solely uses point cloud data as input, similar to the iterative closest point algorithm (ICP). The aim of this study is to automatically classify deformations that occurred at a rock glacier and an ice glacier, as well as in a rockfall area. For every case study, two epochs were processed, while the datasets notably differ in terms of geometric characteristics, distribution and magnitude of deformation. In summary, the ICProx algorithm's classification accuracy is 70 % on average in comparison to reference data.

Automatic Extraction of Road Markings from Mobile Laser-Point Cloud Using Intensity Data

NASA Astrophysics Data System (ADS)

Yao, L.; Chen, Q.; Qin, C.; Wu, H.; Zhang, S.

2018-04-01

With the development of intelligent transportation, road's high precision information data has been widely applied in many fields. This paper proposes a concise and practical way to extract road marking information from point cloud data collected by mobile mapping system (MMS). The method contains three steps. Firstly, road surface is segmented through edge detection from scan lines. Then the intensity image is generated by inverse distance weighted (IDW) interpolation and the road marking is extracted by using adaptive threshold segmentation based on integral image without intensity calibration. Moreover, the noise is reduced by removing a small number of plaque pixels from binary image. Finally, point cloud mapped from binary image is clustered into marking objects according to Euclidean distance, and using a series of algorithms including template matching and feature attribute filtering for the classification of linear markings, arrow markings and guidelines. Through processing the point cloud data collected by RIEGL VUX-1 in case area, the results show that the F-score of marking extraction is 0.83, and the average classification rate is 0.9.

(Un)Natural Disasters: The Electoral Cycle Outweighs the Hydrologic Cycle in Drought Declaration in Northeast Brazil

NASA Astrophysics Data System (ADS)

Camps-Valls, G.; Gomez-Chova, L.; Mateo, G.; Laparra, V.; Perez-Suay, A.; Munoz-Mari, J.

2016-12-01

Current Earth-observation (EO) applications for image classification have to deal with an unprecedented big amount of heterogeneous and complex data sources. Spatio-temporally explicit classification methods are a requirement in a variety of Earth system data processing applications. Upcoming missions such as the super-spectral Copernicus Sentinels EnMAP and FLEX will soon provide unprecedented data streams. Very high resolution (VHR) sensors like Worldview-3 also pose big challenges to data processing. The challenge is not only attached to optical sensors but also to infrared sounders and radar images which increased in spectral, spatial and temporal resolution. Besides, we should not forget the availability of the extremely large remote sensing data archives already collected by several past missions, such ENVISAT, Cosmo-SkyMED, Landsat, SPOT, or Seviri/MSG. These large-scale data problems require enhanced processing techniques that should be accurate, robust and fast. Standard parameter retrieval and classification algorithms cannot cope with this new scenario efficiently. In this work, we review the field of large scale kernel methods for both atmospheric parameter retrieval and cloud detection using infrared sounding IASI data and optical Seviri/MSG imagery. We propose novel Gaussian Processes (GPs) to train problems with millions of instances and high number of input features. Algorithms can cope with non-linearities efficiently, accommodate multi-output problems, and provide confidence intervals for the predictions. Several strategies to speed up algorithms are devised: random Fourier features and variational approaches for cloud classification using IASI data and Seviri/MSG, and engineered randomized kernel functions and emulation in temperature, moisture and ozone atmospheric profile retrieval from IASI as a proxy to the upcoming MTG-IRS sensor. Excellent compromise between accuracy and scalability are obtained in all applications.

Classification of Clouds in Satellite Imagery Using Adaptive Fuzzy Sparse Representation.

PubMed

Jin, Wei; Gong, Fei; Zeng, Xingbin; Fu, Randi

2016-12-16

Automatic cloud detection and classification using satellite cloud imagery have various meteorological applications such as weather forecasting and climate monitoring. Cloud pattern analysis is one of the research hotspots recently. Since satellites sense the clouds remotely from space, and different cloud types often overlap and convert into each other, there must be some fuzziness and uncertainty in satellite cloud imagery. Satellite observation is susceptible to noises, while traditional cloud classification methods are sensitive to noises and outliers; it is hard for traditional cloud classification methods to achieve reliable results. To deal with these problems, a satellite cloud classification method using adaptive fuzzy sparse representation-based classification (AFSRC) is proposed. Firstly, by defining adaptive parameters related to attenuation rate and critical membership, an improved fuzzy membership is introduced to accommodate the fuzziness and uncertainty of satellite cloud imagery; secondly, by effective combination of the improved fuzzy membership function and sparse representation-based classification (SRC), atoms in training dictionary are optimized; finally, an adaptive fuzzy sparse representation classifier for cloud classification is proposed. Experiment results on FY-2G satellite cloud image show that, the proposed method not only improves the accuracy of cloud classification, but also has strong stability and adaptability with high computational efficiency.

Cross-validation pitfalls when selecting and assessing regression and classification models.

PubMed

Krstajic, Damjan; Buturovic, Ljubomir J; Leahy, David E; Thomas, Simon

2014-03-29

We address the problem of selecting and assessing classification and regression models using cross-validation. Current state-of-the-art methods can yield models with high variance, rendering them unsuitable for a number of practical applications including QSAR. In this paper we describe and evaluate best practices which improve reliability and increase confidence in selected models. A key operational component of the proposed methods is cloud computing which enables routine use of previously infeasible approaches. We describe in detail an algorithm for repeated grid-search V-fold cross-validation for parameter tuning in classification and regression, and we define a repeated nested cross-validation algorithm for model assessment. As regards variable selection and parameter tuning we define two algorithms (repeated grid-search cross-validation and double cross-validation), and provide arguments for using the repeated grid-search in the general case. We show results of our algorithms on seven QSAR datasets. The variation of the prediction performance, which is the result of choosing different splits of the dataset in V-fold cross-validation, needs to be taken into account when selecting and assessing classification and regression models. We demonstrate the importance of repeating cross-validation when selecting an optimal model, as well as the importance of repeating nested cross-validation when assessing a prediction error.

Classification of Clouds in Satellite Imagery Using Adaptive Fuzzy Sparse Representation

PubMed Central

Jin, Wei; Gong, Fei; Zeng, Xingbin; Fu, Randi

2016-01-01

Automatic cloud detection and classification using satellite cloud imagery have various meteorological applications such as weather forecasting and climate monitoring. Cloud pattern analysis is one of the research hotspots recently. Since satellites sense the clouds remotely from space, and different cloud types often overlap and convert into each other, there must be some fuzziness and uncertainty in satellite cloud imagery. Satellite observation is susceptible to noises, while traditional cloud classification methods are sensitive to noises and outliers; it is hard for traditional cloud classification methods to achieve reliable results. To deal with these problems, a satellite cloud classification method using adaptive fuzzy sparse representation-based classification (AFSRC) is proposed. Firstly, by defining adaptive parameters related to attenuation rate and critical membership, an improved fuzzy membership is introduced to accommodate the fuzziness and uncertainty of satellite cloud imagery; secondly, by effective combination of the improved fuzzy membership function and sparse representation-based classification (SRC), atoms in training dictionary are optimized; finally, an adaptive fuzzy sparse representation classifier for cloud classification is proposed. Experiment results on FY-2G satellite cloud image show that, the proposed method not only improves the accuracy of cloud classification, but also has strong stability and adaptability with high computational efficiency. PMID:27999261

Classification Algorithms for Big Data Analysis, a Map Reduce Approach

NASA Astrophysics Data System (ADS)

Ayma, V. A.; Ferreira, R. S.; Happ, P.; Oliveira, D.; Feitosa, R.; Costa, G.; Plaza, A.; Gamba, P.

2015-03-01

Since many years ago, the scientific community is concerned about how to increase the accuracy of different classification methods, and major achievements have been made so far. Besides this issue, the increasing amount of data that is being generated every day by remote sensors raises more challenges to be overcome. In this work, a tool within the scope of InterIMAGE Cloud Platform (ICP), which is an open-source, distributed framework for automatic image interpretation, is presented. The tool, named ICP: Data Mining Package, is able to perform supervised classification procedures on huge amounts of data, usually referred as big data, on a distributed infrastructure using Hadoop MapReduce. The tool has four classification algorithms implemented, taken from WEKA's machine learning library, namely: Decision Trees, Naïve Bayes, Random Forest and Support Vector Machines (SVM). The results of an experimental analysis using a SVM classifier on data sets of different sizes for different cluster configurations demonstrates the potential of the tool, as well as aspects that affect its performance.

Thin ice clouds in the Arctic: cloud optical depth and particle size retrieved from ground-based thermal infrared radiometry

DOE Office of Scientific and Technical Information (OSTI.GOV)

Blanchard, Yann; Royer, Alain; O'Neill, Norman T.

Multiband downwelling thermal measurements of zenith sky radiance, along with cloud boundary heights, were used in a retrieval algorithm to estimate cloud optical depth and effective particle diameter of thin ice clouds in the Canadian High Arctic. Ground-based thermal infrared (IR) radiances for 150 semitransparent ice clouds cases were acquired at the Polar Environment Atmospheric Research Laboratory (PEARL) in Eureka, Nunavut, Canada (80° N, 86° W). We analyzed and quantified the sensitivity of downwelling thermal radiance to several cloud parameters including optical depth, effective particle diameter and shape, water vapor content, cloud geometric thickness and cloud base altitude. A lookupmore » table retrieval method was used to successfully extract, through an optimal estimation method, cloud optical depth up to a maximum value of 2.6 and to separate thin ice clouds into two classes: (1) TIC1 clouds characterized by small crystals (effective particle diameter ≤ 30 µm), and (2) TIC2 clouds characterized by large ice crystals (effective particle diameter > 30 µm). The retrieval technique was validated using data from the Arctic High Spectral Resolution Lidar (AHSRL) and Millimeter Wave Cloud Radar (MMCR). Inversions were performed over three polar winters and results showed a significant correlation ( R 2 = 0.95) for cloud optical depth retrievals and an overall accuracy of 83 % for the classification of TIC1 and TIC2 clouds. A partial validation relative to an algorithm based on high spectral resolution downwelling IR radiance measurements between 8 and 21µm was also performed. It confirms the robustness of the optical depth retrieval and the fact that the broadband thermal radiometer retrieval was sensitive to small particle (TIC1) sizes.« less

Thin ice clouds in the Arctic: cloud optical depth and particle size retrieved from ground-based thermal infrared radiometry

NASA Astrophysics Data System (ADS)

Blanchard, Yann; Royer, Alain; O'Neill, Norman T.; Turner, David D.; Eloranta, Edwin W.

2017-06-01

Multiband downwelling thermal measurements of zenith sky radiance, along with cloud boundary heights, were used in a retrieval algorithm to estimate cloud optical depth and effective particle diameter of thin ice clouds in the Canadian High Arctic. Ground-based thermal infrared (IR) radiances for 150 semitransparent ice clouds cases were acquired at the Polar Environment Atmospheric Research Laboratory (PEARL) in Eureka, Nunavut, Canada (80° N, 86° W). We analyzed and quantified the sensitivity of downwelling thermal radiance to several cloud parameters including optical depth, effective particle diameter and shape, water vapor content, cloud geometric thickness and cloud base altitude. A lookup table retrieval method was used to successfully extract, through an optimal estimation method, cloud optical depth up to a maximum value of 2.6 and to separate thin ice clouds into two classes: (1) TIC1 clouds characterized by small crystals (effective particle diameter ≤ 30 µm), and (2) TIC2 clouds characterized by large ice crystals (effective particle diameter > 30 µm). The retrieval technique was validated using data from the Arctic High Spectral Resolution Lidar (AHSRL) and Millimeter Wave Cloud Radar (MMCR). Inversions were performed over three polar winters and results showed a significant correlation (R2 = 0.95) for cloud optical depth retrievals and an overall accuracy of 83 % for the classification of TIC1 and TIC2 clouds. A partial validation relative to an algorithm based on high spectral resolution downwelling IR radiance measurements between 8 and 21 µm was also performed. It confirms the robustness of the optical depth retrieval and the fact that the broadband thermal radiometer retrieval was sensitive to small particle (TIC1) sizes.

Thin ice clouds in the Arctic: cloud optical depth and particle size retrieved from ground-based thermal infrared radiometry

DOE PAGES

Blanchard, Yann; Royer, Alain; O'Neill, Norman T.; ...

2017-06-09

Multiband downwelling thermal measurements of zenith sky radiance, along with cloud boundary heights, were used in a retrieval algorithm to estimate cloud optical depth and effective particle diameter of thin ice clouds in the Canadian High Arctic. Ground-based thermal infrared (IR) radiances for 150 semitransparent ice clouds cases were acquired at the Polar Environment Atmospheric Research Laboratory (PEARL) in Eureka, Nunavut, Canada (80° N, 86° W). We analyzed and quantified the sensitivity of downwelling thermal radiance to several cloud parameters including optical depth, effective particle diameter and shape, water vapor content, cloud geometric thickness and cloud base altitude. A lookupmore » table retrieval method was used to successfully extract, through an optimal estimation method, cloud optical depth up to a maximum value of 2.6 and to separate thin ice clouds into two classes: (1) TIC1 clouds characterized by small crystals (effective particle diameter ≤ 30 µm), and (2) TIC2 clouds characterized by large ice crystals (effective particle diameter > 30 µm). The retrieval technique was validated using data from the Arctic High Spectral Resolution Lidar (AHSRL) and Millimeter Wave Cloud Radar (MMCR). Inversions were performed over three polar winters and results showed a significant correlation ( R 2 = 0.95) for cloud optical depth retrievals and an overall accuracy of 83 % for the classification of TIC1 and TIC2 clouds. A partial validation relative to an algorithm based on high spectral resolution downwelling IR radiance measurements between 8 and 21µm was also performed. It confirms the robustness of the optical depth retrieval and the fact that the broadband thermal radiometer retrieval was sensitive to small particle (TIC1) sizes.« less

An Algorithm to Identify and Localize Suitable Dock Locations from 3-D LiDAR Scans

DTIC Science & Technology

2013-05-10

Locations from 3-D LiDAR Scans 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) Graves, Mitchell Robert 5d. PROJECT NUMBER...Ranging ( LiDAR ) scans. A LiDAR sensor is a sensor that collects range images from a rotating array of vertically aligned lasers. Our solution leverages...Algorithm, Dock, Locations, Point Clouds, LiDAR , Identify 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT 18. NUMBER OF PAGES 19a

A satellite rainfall retrieval technique over northern Algeria based on the probability of rainfall intensities classification from MSG-SEVIRI

NASA Astrophysics Data System (ADS)

Lazri, Mourad; Ameur, Soltane

2016-09-01

In this paper, an algorithm based on the probability of rainfall intensities classification for rainfall estimation from Meteosat Second Generation/Spinning Enhanced Visible and Infrared Imager (MSG-SEVIRI) has been developed. The classification scheme uses various spectral parameters of SEVIRI that provide information about cloud top temperature and optical and microphysical cloud properties. The presented method is developed and trained for the north of Algeria. The calibration of the method is carried out using as a reference rain classification fields derived from radar for rainy season from November 2006 to March 2007. Rainfall rates are assigned to rain areas previously identified and classified according to the precipitation formation processes. The comparisons between satellite-derived precipitation estimates and validation data show that the developed scheme performs reasonably well. Indeed, the correlation coefficient presents a significant level (r:0.87). The values of POD, POFD and FAR are 80%, 13% and 25%, respectively. Also, for a rainfall estimation of about 614 mm, the RMSD, Bias, MAD and PD indicate 102.06(mm), 2.18(mm), 68.07(mm) and 12.58, respectively.

Lidar Cloud Detection with Fully Convolutional Networks

NASA Astrophysics Data System (ADS)

Cromwell, E.; Flynn, D.

2017-12-01

The vertical distribution of clouds from active remote sensing instrumentation is a widely used data product from global atmospheric measuring sites. The presence of clouds can be expressed as a binary cloud mask and is a primary input for climate modeling efforts and cloud formation studies. Current cloud detection algorithms producing these masks do not accurately identify the cloud boundaries and tend to oversample or over-represent the cloud. This translates as uncertainty for assessing the radiative impact of clouds and tracking changes in cloud climatologies. The Atmospheric Radiation Measurement (ARM) program has over 20 years of micro-pulse lidar (MPL) and High Spectral Resolution Lidar (HSRL) instrument data and companion automated cloud mask product at the mid-latitude Southern Great Plains (SGP) and the polar North Slope of Alaska (NSA) atmospheric observatory. Using this data, we train a fully convolutional network (FCN) with semi-supervised learning to segment lidar imagery into geometric time-height cloud locations for the SGP site and MPL instrument. We then use transfer learning to train a FCN for (1) the MPL instrument at the NSA site and (2) for the HSRL. In our semi-supervised approach, we pre-train the classification layers of the FCN with weakly labeled lidar data. Then, we facilitate end-to-end unsupervised pre-training and transition to fully supervised learning with ground truth labeled data. Our goal is to improve the cloud mask accuracy and precision for the MPL instrument to 95% and 80%, respectively, compared to the current cloud mask algorithms of 89% and 50%. For the transfer learning based FCN for the HSRL instrument, our goal is to achieve a cloud mask accuracy of 90% and a precision of 80%.

Automated cloud screening of AVHRR imagery using split-and-merge clustering

NASA Technical Reports Server (NTRS)

Gallaudet, Timothy C.; Simpson, James J.

1991-01-01

Previous methods to segment clouds from ocean in AVHRR imagery have shown varying degrees of success, with nighttime approaches being the most limited. An improved method of automatic image segmentation, the principal component transformation split-and-merge clustering (PCTSMC) algorithm, is presented and applied to cloud screening of both nighttime and daytime AVHRR data. The method combines spectral differencing, the principal component transformation, and split-and-merge clustering to sample objectively the natural classes in the data. This segmentation method is then augmented by supervised classification techniques to screen clouds from the imagery. Comparisons with other nighttime methods demonstrate its improved capability in this application. The sensitivity of the method to clustering parameters is presented; the results show that the method is insensitive to the split-and-merge thresholds.

A Method to Estimate Sunshine Duration Using Cloud Classification Data from a Geostationary Meteorological Satellite (FY-2D) over the Heihe River Basin.

PubMed

Wu, Bingfang; Liu, Shufu; Zhu, Weiwei; Yu, Mingzhao; Yan, Nana; Xing, Qiang

2016-11-04

Sunshine duration is an important variable that is widely used in atmospheric energy balance studies, analysis of the thermal loadings on buildings, climate research, and the evaluation of agricultural resources. In most cases, it is calculated using an interpolation method based on regional-scale meteorological data from field stations. Accurate values in the field are difficult to obtain without ground measurements. In this paper, a satellite-based method to estimate sunshine duration is introduced and applied over the Heihe River Basin. This method is based on hourly cloud classification product data from the FY-2D geostationary meteorological satellite (FY-2D). A new index-FY-2D cloud type sunshine factor-is proposed, and the Shuffled Complex Evolution Algorithm (SCE-UA) was used to calibrate sunshine factors from different coverage types based on ground measurement data from the Heihe River Basin in 2007. The estimated sunshine duration from the proposed new algorithm was validated with ground observation data for 12 months in 2008, and the spatial distribution was compared with the results of an interpolation method over the Heihe River Basin. The study demonstrates that geostationary satellite data can be used to successfully estimate sunshine duration. Potential applications include climate research, energy balance studies, and global estimations of evapotranspiration.

A Method to Estimate Sunshine Duration Using Cloud Classification Data from a Geostationary Meteorological Satellite (FY-2D) over the Heihe River Basin

PubMed Central

Wu, Bingfang; Liu, Shufu; Zhu, Weiwei; Yu, Mingzhao; Yan, Nana; Xing, Qiang

2016-01-01

Sunshine duration is an important variable that is widely used in atmospheric energy balance studies, analysis of the thermal loadings on buildings, climate research, and the evaluation of agricultural resources. In most cases, it is calculated using an interpolation method based on regional-scale meteorological data from field stations. Accurate values in the field are difficult to obtain without ground measurements. In this paper, a satellite-based method to estimate sunshine duration is introduced and applied over the Heihe River Basin. This method is based on hourly cloud classification product data from the FY-2D geostationary meteorological satellite (FY-2D). A new index—FY-2D cloud type sunshine factor—is proposed, and the Shuffled Complex Evolution Algorithm (SCE-UA) was used to calibrate sunshine factors from different coverage types based on ground measurement data from the Heihe River Basin in 2007. The estimated sunshine duration from the proposed new algorithm was validated with ground observation data for 12 months in 2008, and the spatial distribution was compared with the results of an interpolation method over the Heihe River Basin. The study demonstrates that geostationary satellite data can be used to successfully estimate sunshine duration. Potential applications include climate research, energy balance studies, and global estimations of evapotranspiration. PMID:27827935

Cloud and DNI nowcasting with MSG/SEVIRI for the optimized operation of concentrating solar power plants

NASA Astrophysics Data System (ADS)

Sirch, Tobias; Bugliaro, Luca; Zinner, Tobias; Möhrlein, Matthias; Vazquez-Navarro, Margarita

2017-02-01

A novel approach for the nowcasting of clouds and direct normal irradiance (DNI) based on the Spinning Enhanced Visible and Infrared Imager (SEVIRI) aboard the geostationary Meteosat Second Generation (MSG) satellite is presented for a forecast horizon up to 120 min. The basis of the algorithm is an optical flow method to derive cloud motion vectors for all cloudy pixels. To facilitate forecasts over a relevant time period, a classification of clouds into objects and a weighted triangular interpolation of clear-sky regions are used. Low and high level clouds are forecasted separately because they show different velocities and motion directions. Additionally a distinction in advective and convective clouds together with an intensity correction for quickly thinning convective clouds is integrated. The DNI is calculated from the forecasted optical thickness of the low and high level clouds. In order to quantitatively assess the performance of the algorithm, a forecast validation against MSG/SEVIRI observations is performed for a period of 2 months. Error rates and Hanssen-Kuiper skill scores are derived for forecasted cloud masks. For a forecast of 5 min for most cloud situations more than 95 % of all pixels are predicted correctly cloudy or clear. This number decreases to 80-95 % for a forecast of 2 h depending on cloud type and vertical cloud level. Hanssen-Kuiper skill scores for cloud mask go down to 0.6-0.7 for a 2 h forecast. Compared to persistence an improvement of forecast horizon by a factor of 2 is reached for all forecasts up to 2 h. A comparison of forecasted optical thickness distributions and DNI against observations yields correlation coefficients larger than 0.9 for 15 min forecasts and around 0.65 for 2 h forecasts.

Sentinel-2 Level 2A Prototype Processor: Architecture, Algorithms And First Results

NASA Astrophysics Data System (ADS)

Muller-Wilm, Uwe; Louis, Jerome; Richter, Rudolf; Gascon, Ferran; Niezette, Marc

2013-12-01

Sen2Core is a prototype processor for Sentinel-2 Level 2A product processing and formatting. The processor is developed for and with ESA and performs the tasks of Atmospheric Correction and Scene Classification of Level 1C input data. Level 2A outputs are: Bottom-Of- Atmosphere (BOA) corrected reflectance images, Aerosol Optical Thickness-, Water Vapour-, Scene Classification maps and Quality indicators, including cloud and snow probabilities. The Level 2A Product Formatting performed by the processor follows the specification of the Level 1C User Product.

A Novel Anti-classification Approach for Knowledge Protection.

PubMed

Lin, Chen-Yi; Chen, Tung-Shou; Tsai, Hui-Fang; Lee, Wei-Bin; Hsu, Tien-Yu; Kao, Yuan-Hung

2015-10-01

Classification is the problem of identifying a set of categories where new data belong, on the basis of a set of training data whose category membership is known. Its application is wide-spread, such as the medical science domain. The issue of the classification knowledge protection has been paid attention increasingly in recent years because of the popularity of cloud environments. In the paper, we propose a Shaking Sorted-Sampling (triple-S) algorithm for protecting the classification knowledge of a dataset. The triple-S algorithm sorts the data of an original dataset according to the projection results of the principal components analysis so that the features of the adjacent data are similar. Then, we generate noise data with incorrect classes and add those data to the original dataset. In addition, we develop an effective positioning strategy, determining the added positions of noise data in the original dataset, to ensure the restoration of the original dataset after removing those noise data. The experimental results show that the disturbance effect of the triple-S algorithm on the CLC, MySVM, and LibSVM classifiers increases when the noise data ratio increases. In addition, compared with existing methods, the disturbance effect of the triple-S algorithm is more significant on MySVM and LibSVM when a certain amount of the noise data added to the original dataset is reached.

Foliar and woody materials discriminated using terrestrial LiDAR in a mixed natural forest

NASA Astrophysics Data System (ADS)

Zhu, Xi; Skidmore, Andrew K.; Darvishzadeh, Roshanak; Niemann, K. Olaf; Liu, Jing; Shi, Yifang; Wang, Tiejun

2018-02-01

Separation of foliar and woody materials using remotely sensed data is crucial for the accurate estimation of leaf area index (LAI) and woody biomass across forest stands. In this paper, we present a new method to accurately separate foliar and woody materials using terrestrial LiDAR point clouds obtained from ten test sites in a mixed forest in Bavarian Forest National Park, Germany. Firstly, we applied and compared an adaptive radius near-neighbor search algorithm with a fixed radius near-neighbor search method in order to obtain both radiometric and geometric features derived from terrestrial LiDAR point clouds. Secondly, we used a random forest machine learning algorithm to classify foliar and woody materials and examined the impact of understory and slope on the classification accuracy. An average overall accuracy of 84.4% (Kappa = 0.75) was achieved across all experimental plots. The adaptive radius near-neighbor search method outperformed the fixed radius near-neighbor search method. The classification accuracy was significantly higher when the combination of both radiometric and geometric features was utilized. The analysis showed that increasing slope and understory coverage had a significant negative effect on the overall classification accuracy. Our results suggest that the utilization of the adaptive radius near-neighbor search method coupling both radiometric and geometric features has the potential to accurately discriminate foliar and woody materials from terrestrial LiDAR data in a mixed natural forest.

Object-Based Point Cloud Analysis of Full-Waveform Airborne Laser Scanning Data for Urban Vegetation Classification

PubMed Central

Rutzinger, Martin; Höfle, Bernhard; Hollaus, Markus; Pfeifer, Norbert

2008-01-01

Airborne laser scanning (ALS) is a remote sensing technique well-suited for 3D vegetation mapping and structure characterization because the emitted laser pulses are able to penetrate small gaps in the vegetation canopy. The backscattered echoes from the foliage, woody vegetation, the terrain, and other objects are detected, leading to a cloud of points. Higher echo densities (>20 echoes/m2) and additional classification variables from full-waveform (FWF) ALS data, namely echo amplitude, echo width and information on multiple echoes from one shot, offer new possibilities in classifying the ALS point cloud. Currently FWF sensor information is hardly used for classification purposes. This contribution presents an object-based point cloud analysis (OBPA) approach, combining segmentation and classification of the 3D FWF ALS points designed to detect tall vegetation in urban environments. The definition tall vegetation includes trees and shrubs, but excludes grassland and herbage. In the applied procedure FWF ALS echoes are segmented by a seeded region growing procedure. All echoes sorted descending by their surface roughness are used as seed points. Segments are grown based on echo width homogeneity. Next, segment statistics (mean, standard deviation, and coefficient of variation) are calculated by aggregating echo features such as amplitude and surface roughness. For classification a rule base is derived automatically from a training area using a statistical classification tree. To demonstrate our method we present data of three sites with around 500,000 echoes each. The accuracy of the classified vegetation segments is evaluated for two independent validation sites. In a point-wise error assessment, where the classification is compared with manually classified 3D points, completeness and correctness better than 90% are reached for the validation sites. In comparison to many other algorithms the proposed 3D point classification works on the original measurements directly, i.e. the acquired points. Gridding of the data is not necessary, a process which is inherently coupled to loss of data and precision. The 3D properties provide especially a good separability of buildings and terrain points respectively, if they are occluded by vegetation. PMID:27873771

Evaluation of Decision Trees for Cloud Detection from AVHRR Data

NASA Technical Reports Server (NTRS)

Shiffman, Smadar; Nemani, Ramakrishna

2005-01-01

Automated cloud detection and tracking is an important step in assessing changes in radiation budgets associated with global climate change via remote sensing. Data products based on satellite imagery are available to the scientific community for studying trends in the Earth's atmosphere. The data products include pixel-based cloud masks that assign cloud-cover classifications to pixels. Many cloud-mask algorithms have the form of decision trees. The decision trees employ sequential tests that scientists designed based on empirical astrophysics studies and simulations. Limitations of existing cloud masks restrict our ability to accurately track changes in cloud patterns over time. In a previous study we compared automatically learned decision trees to cloud masks included in Advanced Very High Resolution Radiometer (AVHRR) data products from the year 2000. In this paper we report the replication of the study for five-year data, and for a gold standard based on surface observations performed by scientists at weather stations in the British Islands. For our sample data, the accuracy of automatically learned decision trees was greater than the accuracy of the cloud masks p < 0.001.

Crop classification and mapping based on Sentinel missions data in cloud environment

NASA Astrophysics Data System (ADS)

Lavreniuk, M. S.; Kussul, N.; Shelestov, A.; Vasiliev, V.

2017-12-01

Availability of high resolution satellite imagery (Sentinel-1/2/3, Landsat) over large territories opens new opportunities in agricultural monitoring. In particular, it becomes feasible to solve crop classification and crop mapping task at country and regional scale using time series of heterogenous satellite imagery. But in this case, we face with the problem of Big Data. Dealing with time series of high resolution (10 m) multispectral imagery we need to download huge volumes of data and then process them. The solution is to move "processing chain" closer to data itself to drastically shorten time for data transfer. One more advantage of such approach is the possibility to parallelize data processing workflow and efficiently implement machine learning algorithms. This could be done with cloud platform where Sentinel imagery are stored. In this study, we investigate usability and efficiency of two different cloud platforms Amazon and Google for crop classification and crop mapping problems. Two pilot areas were investigated - Ukraine and England. Google provides user friendly environment Google Earth Engine for Earth observation applications with a lot of data processing and machine learning tools already deployed. At the same time with Amazon one gets much more flexibility in implementation of his own workflow. Detailed analysis of pros and cons will be done in the presentation.

A combined spectral and object-based approach to transparent cloud removal in an operational setting for Landsat ETM+

NASA Astrophysics Data System (ADS)

Watmough, Gary R.; Atkinson, Peter M.; Hutton, Craig W.

2011-04-01

The automated cloud cover assessment (ACCA) algorithm has provided automated estimates of cloud cover for the Landsat ETM+ mission since 2001. However, due to the lack of a band around 1.375 μm, cloud edges and transparent clouds such as cirrus cannot be detected. Use of Landsat ETM+ imagery for terrestrial land analysis is further hampered by the relatively long revisit period due to a nadir only viewing sensor. In this study, the ACCA threshold parameters were altered to minimise omission errors in the cloud masks. Object-based analysis was used to reduce the commission errors from the extended cloud filters. The method resulted in the removal of optically thin cirrus cloud and cloud edges which are often missed by other methods in sub-tropical areas. Although not fully automated, the principles of the method developed here provide an opportunity for using otherwise sub-optimal or completely unusable Landsat ETM+ imagery for operational applications. Where specific images are required for particular research goals the method can be used to remove cloud and transparent cloud helping to reduce bias in subsequent land cover classifications.

Precipitation Estimation from Remotely Sensed Information using Artificial Neural Network-Cloud Classification System

NASA Astrophysics Data System (ADS)

Hong, Yang

Precipitation estimation from satellite information (VISIBLE , IR, or microwave) is becoming increasingly imperative because of its high spatial/temporal resolution and board coverage unparalleled by ground-based data. After decades' efforts of rainfall estimation using IR imagery as basis, it has been explored and concluded that the limitations/uncertainty of the existing techniques are: (1) pixel-based local-scale feature extraction; (2) IR temperature threshold to define rain/no-rain clouds; (3) indirect relationship between rain rate and cloud-top temperature; (4) lumped techniques to model high variability of cloud-precipitation processes; (5) coarse scales of rainfall products. As continuing studies, a new version of Precipitation Estimation from Remotely Sensed Information using Artificial Neural Network (PERSIANN), called Cloud Classification System (CCS), has been developed to cope with these limitations in this dissertation. CCS includes three consecutive components: (1) a hybrid segmentation algorithm, namely Hierarchically Topographical Thresholding and Stepwise Seeded Region Growing (HTH-SSRG), to segment satellite IR images into separated cloud patches; (2) a 3D feature extraction procedure to retrieve both pixel-based local-scale and patch-based large-scale features of cloud patch at various heights; (3) an ANN model, Self-Organizing Nonlinear Output (SONO) network, to classify cloud patches into similarity-based clusters, using Self-Organizing Feature Map (SOFM), and then calibrate hundreds of multi-parameter nonlinear functions to identify the relationship between every cloud types and their underneath precipitation characteristics using Probability Matching Method and Multi-Start Downhill Simplex optimization techniques. The model was calibrated over the Southwest of United States (100°--130°W and 25°--45°N) first and then adaptively adjusted to the study region of North America Monsoon Experiment (65°--135°W and 10°--50°N) using observations from Geostationary Operational Environmental Satellite (GOES) IR imagery, Next Generation Radar (NEXRAD) rainfall network, and Tropical Rainfall Measurement Mission (TRMM) microwave rain rate estimates. CCS functions as a distributed model that first identifies cloud patches and then dispatches different but the best matching cloud-precipitation function for each cloud patch to estimate instantaneous rain rate at high spatial resolution (4km) and full temporal resolution of GOES IR images (every 30-minute). Evaluated over a range of spatial and temporal scales, the performance of CCS compared favorably with GOES Precipitation Index (GPI), Universal Adjusted GPI (UAGPI), PERSIANN, and Auto-Estimator (AE) algorithms, consistently. Particularly, the large number of nonlinear functions and optimum IR-rain rate thresholds of CCS model are highly variable, reflecting the complexity of dominant cloud-precipitation processes from cloud patch to cloud patch over various regions. As a result, CCS can more successfully capture variability in rain rate at small scales than existing algorithms and potentially provides rainfall product from GOES IR-NEXARD-TRMM TMI (SSM/I) at 0.12° x 0.12° and 3-hour resolution with relative low standard error (˜=3.0mm/hr) and high correlation coefficient (˜=0.65).

Cloud Classification in Polar and Desert Regions and Smoke Classification from Biomass Burning Using a Hierarchical Neural Network

NASA Technical Reports Server (NTRS)

Alexander, June; Corwin, Edward; Lloyd, David; Logar, Antonette; Welch, Ronald

1996-01-01

This research focuses on a new neural network scene classification technique. The task is to identify scene elements in Advanced Very High Resolution Radiometry (AVHRR) data from three scene types: polar, desert and smoke from biomass burning in South America (smoke). The ultimate goal of this research is to design and implement a computer system which will identify the clouds present on a whole-Earth satellite view as a means of tracking global climate changes. Previous research has reported results for rule-based systems (Tovinkere et at 1992, 1993) for standard back propagation (Watters et at. 1993) and for a hierarchical approach (Corwin et al 1994) for polar data. This research uses a hierarchical neural network with don't care conditions and applies this technique to complex scenes. A hierarchical neural network consists of a switching network and a collection of leaf networks. The idea of the hierarchical neural network is that it is a simpler task to classify a certain pattern from a subset of patterns than it is to classify a pattern from the entire set. Therefore, the first task is to cluster the classes into groups. The switching, or decision network, performs an initial classification by selecting a leaf network. The leaf networks contain a reduced set of similar classes, and it is in the various leaf networks that the actual classification takes place. The grouping of classes in the various leaf networks is determined by applying an iterative clustering algorithm. Several clustering algorithms were investigated, but due to the size of the data sets, the exhaustive search algorithms were eliminated. A heuristic approach using a confusion matrix from a lightly trained neural network provided the basis for the clustering algorithm. Once the clusters have been identified, the hierarchical network can be trained. The approach of using don't care nodes results from the difficulty in generating extremely complex surfaces in order to separate one class from all of the others. This approach finds pairwise separating surfaces and forms the more complex separating surface from combinations of simpler surfaces. This technique both reduces training time and improves accuracy over the previously reported results. Accuracies of 97.47%, 95.70%, and 99.05% were achieved for the polar, desert and smoke data sets.

Georeferencing UAS Derivatives Through Point Cloud Registration with Archived Lidar Datasets

NASA Astrophysics Data System (ADS)

Magtalas, M. S. L. Y.; Aves, J. C. L.; Blanco, A. C.

2016-10-01

Georeferencing gathered images is a common step before performing spatial analysis and other processes on acquired datasets using unmanned aerial systems (UAS). Methods of applying spatial information to aerial images or their derivatives is through onboard GPS (Global Positioning Systems) geotagging, or through tying of models through GCPs (Ground Control Points) acquired in the field. Currently, UAS (Unmanned Aerial System) derivatives are limited to meter-levels of accuracy when their generation is unaided with points of known position on the ground. The use of ground control points established using survey-grade GPS or GNSS receivers can greatly reduce model errors to centimeter levels. However, this comes with additional costs not only with instrument acquisition and survey operations, but also in actual time spent in the field. This study uses a workflow for cloud-based post-processing of UAS data in combination with already existing LiDAR data. The georeferencing of the UAV point cloud is executed using the Iterative Closest Point algorithm (ICP). It is applied through the open-source CloudCompare software (Girardeau-Montaut, 2006) on a `skeleton point cloud'. This skeleton point cloud consists of manually extracted features consistent on both LiDAR and UAV data. For this cloud, roads and buildings with minimal deviations given their differing dates of acquisition are considered consistent. Transformation parameters are computed for the skeleton cloud which could then be applied to the whole UAS dataset. In addition, a separate cloud consisting of non-vegetation features automatically derived using CANUPO classification algorithm (Brodu and Lague, 2012) was used to generate a separate set of parameters. Ground survey is done to validate the transformed cloud. An RMSE value of around 16 centimeters was found when comparing validation data to the models georeferenced using the CANUPO cloud and the manual skeleton cloud. Cloud-to-cloud distance computations of CANUPO and manual skeleton clouds were obtained with values for both equal to around 0.67 meters at 1.73 standard deviation.

Enhancement and evaluation of an algorithm for atmospheric profiling continuity from Aqua to Suomi-NPP

NASA Astrophysics Data System (ADS)

Lipton, A.; Moncet, J. L.; Payne, V.; Lynch, R.; Polonsky, I. N.

2017-12-01

We will present recent results from an algorithm for producing climate-quality atmospheric profiling earth system data records (ESDRs) for application to data from hyperspectral sounding instruments, including the Atmospheric InfraRed Sounder (AIRS) on EOS Aqua and the Cross-track Infrared Sounder (CrIS) on Suomi-NPP, along with their companion microwave sounders, AMSU and ATMS, respectively. The ESDR algorithm uses an optimal estimation approach and the implementation has a flexible, modular software structure to support experimentation and collaboration. Data record continuity benefits from the fact that the same algorithm can be applied to different sensors, simply by providing suitable configuration and data files. Developments to be presented include the impact of a radiance-based pre-classification method for the atmospheric background. In addition to improving retrieval performance, pre-classification has the potential to reduce the sensitivity of the retrievals to the climatological data from which the background estimate and its error covariance are derived. We will also discuss evaluation of a method for mitigating the effect of clouds on the radiances, and enhancements of the radiative transfer forward model.

Integrated Efforts for Analysis of Geophysical Measurements and Models.

DTIC Science & Technology

1997-09-26

12b. DISTRIBUTION CODE 13. ABSTRACT ( Maximum 200 words) This contract supported investigations of integrated applications of physics, ephemerides...REGIONS AND GPS DATA VALIDATIONS 20 2.5 PL-SCINDA: VISUALIZATION AND ANALYSIS TECHNIQUES 22 2.5.1 View Controls 23 2.5.2 Map Selection...and IR data, about cloudy pixels. Clustering and maximum likelihood classification algorithms categorize up to four cloud layers into stratiform or

Filling of Cloud-Induced Gaps for Land Use and Land Cover Classifications Around Refugee Camps

NASA Astrophysics Data System (ADS)

Braun, Andreas; Hagensieker, Ron; Hochschild, Volker

2016-08-01

Clouds cover is one of the main constraints in the field of optical remote sensing. Especially the use of multispectral imagery is affected by either fully obscured data or parts of the image which remain unusable. This study compares four algorithms for the filling of cloud induced gaps in classified land cover products based on Markov Random Fields (MRF), Random Forest (RF), Closest Spectral Fit (CSF) operators. They are tested on a classified image of Sentinel-2 where artificial clouds are filled by information derived from a scene of Sentinel-1. The approaches rely on different mathematical principles and therefore produced results varying in both pattern and quality. Overall accuracies for the filled areas range from 57 to 64 %. Best results are achieved by CSF, however some classes (e.g. sands and grassland) remain critical through all approaches.

Feature extraction and classification of clouds in high resolution panchromatic satellite imagery

NASA Astrophysics Data System (ADS)

Sharghi, Elan

The development of sophisticated remote sensing sensors is rapidly increasing, and the vast amount of satellite imagery collected is too much to be analyzed manually by a human image analyst. It has become necessary for a tool to be developed to automate the job of an image analyst. This tool would need to intelligently detect and classify objects of interest through computer vision algorithms. Existing software called the Rapid Image Exploitation Resource (RAPIER®) was designed by engineers at Space and Naval Warfare Systems Center Pacific (SSC PAC) to perform exactly this function. This software automatically searches for anomalies in the ocean and reports the detections as a possible ship object. However, if the image contains a high percentage of cloud coverage, a high number of false positives are triggered by the clouds. The focus of this thesis is to explore various feature extraction and classification methods to accurately distinguish clouds from ship objects. An examination of a texture analysis method, line detection using the Hough transform, and edge detection using wavelets are explored as possible feature extraction methods. The features are then supplied to a K-Nearest Neighbors (KNN) or Support Vector Machine (SVM) classifier. Parameter options for these classifiers are explored and the optimal parameters are determined.

Improving PERSIANN-CCS rain estimation using probabilistic approach and multi-sensors information

NASA Astrophysics Data System (ADS)

Karbalaee, N.; Hsu, K. L.; Sorooshian, S.; Kirstetter, P.; Hong, Y.

2016-12-01

This presentation discusses the recent implemented approaches to improve the rainfall estimation from Precipitation Estimation from Remotely Sensed Information using Artificial Neural Network-Cloud Classification System (PERSIANN-CCS). PERSIANN-CCS is an infrared (IR) based algorithm being integrated in the IMERG (Integrated Multi-Satellite Retrievals for the Global Precipitation Mission GPM) to create a precipitation product in 0.1x0.1degree resolution over the chosen domain 50N to 50S every 30 minutes. Although PERSIANN-CCS has a high spatial and temporal resolution, it overestimates or underestimates due to some limitations.PERSIANN-CCS can estimate rainfall based on the extracted information from IR channels at three different temperature threshold levels (220, 235, and 253k). This algorithm relies only on infrared data to estimate rainfall indirectly from this channel which cause missing the rainfall from warm clouds and false estimation for no precipitating cold clouds. In this research the effectiveness of using other channels of GOES satellites such as visible and water vapors has been investigated. By using multi-sensors the precipitation can be estimated based on the extracted information from multiple channels. Also, instead of using the exponential function for estimating rainfall from cloud top temperature, the probabilistic method has been used. Using probability distributions of precipitation rates instead of deterministic values has improved the rainfall estimation for different type of clouds.

Recent Observations of Clouds and Precipitation by the Airborne Precipitation Radar 2nd Generation in Support of the GPM and ACE Missions

NASA Technical Reports Server (NTRS)

Durden, Stephen L.; Tanelli, Simone; Im, Eastwood

2012-01-01

In this paper we illustrate the unique dataset collected during the Global Precipitation Measurement Cold-season Precipitation Experiment (GCPEx, US/Canada Jan/Feb 2012). We will focus on the significance of these observations for the development of algorithms for GPM and ACE, with particular attention to classification and retrievals of frozen and mixed phase hydrometeors.

An automated cirrus classification

NASA Astrophysics Data System (ADS)

Gryspeerdt, Edward; Quaas, Johannes; Sourdeval, Odran; Goren, Tom

2017-04-01

Cirrus clouds play an important role in determining the radiation budget of the earth, but our understanding of the lifecycle and controls on cirrus clouds remains incomplete. Cirrus clouds can have very different properties and development depending on their environment, particularly during their formation. However, the relevant factors often cannot be distinguished using commonly retrieved satellite data products (such as cloud optical depth). In particular, the initial cloud phase has been identified as an important factor in cloud development, but although back-trajectory based methods can provide information on the initial cloud phase, they are computationally expensive and depend on the cloud parametrisations used in re-analysis products. In this work, a classification system (Identification and Classification of Cirrus, IC-CIR) is introduced. Using re-analysis and satellite data, cirrus clouds are separated in four main types: frontal, convective, orographic and in-situ. The properties of these classes show that this classification is able to provide useful information on the properties and initial phase of cirrus clouds, information that could not be provided by instantaneous satellite retrieved cloud properties alone. This classification is designed to be easily implemented in global climate models, helping to improve future comparisons between observations and models and reducing the uncertainty in cirrus clouds properties, leading to improved cloud parametrisations.

A comparison of performance of automatic cloud coverage assessment algorithm for Formosat-2 image using clustering-based and spatial thresholding methods

NASA Astrophysics Data System (ADS)

Hsu, Kuo-Hsien

2012-11-01

Formosat-2 image is a kind of high-spatial-resolution (2 meters GSD) remote sensing satellite data, which includes one panchromatic band and four multispectral bands (Blue, Green, Red, near-infrared). An essential sector in the daily processing of received Formosat-2 image is to estimate the cloud statistic of image using Automatic Cloud Coverage Assessment (ACCA) algorithm. The information of cloud statistic of image is subsequently recorded as an important metadata for image product catalog. In this paper, we propose an ACCA method with two consecutive stages: preprocessing and post-processing analysis. For pre-processing analysis, the un-supervised K-means classification, Sobel's method, thresholding method, non-cloudy pixels reexamination, and cross-band filter method are implemented in sequence for cloud statistic determination. For post-processing analysis, Box-Counting fractal method is implemented. In other words, the cloud statistic is firstly determined via pre-processing analysis, the correctness of cloud statistic of image of different spectral band is eventually cross-examined qualitatively and quantitatively via post-processing analysis. The selection of an appropriate thresholding method is very critical to the result of ACCA method. Therefore, in this work, We firstly conduct a series of experiments of the clustering-based and spatial thresholding methods that include Otsu's, Local Entropy(LE), Joint Entropy(JE), Global Entropy(GE), and Global Relative Entropy(GRE) method, for performance comparison. The result shows that Otsu's and GE methods both perform better than others for Formosat-2 image. Additionally, our proposed ACCA method by selecting Otsu's method as the threshoding method has successfully extracted the cloudy pixels of Formosat-2 image for accurate cloud statistic estimation.

A multiresolution hierarchical classification algorithm for filtering airborne LiDAR data

NASA Astrophysics Data System (ADS)

Chen, Chuanfa; Li, Yanyan; Li, Wei; Dai, Honglei

2013-08-01

We presented a multiresolution hierarchical classification (MHC) algorithm for differentiating ground from non-ground LiDAR point cloud based on point residuals from the interpolated raster surface. MHC includes three levels of hierarchy, with the simultaneous increase of cell resolution and residual threshold from the low to the high level of the hierarchy. At each level, the surface is iteratively interpolated towards the ground using thin plate spline (TPS) until no ground points are classified, and the classified ground points are used to update the surface in the next iteration. 15 groups of benchmark dataset, provided by the International Society for Photogrammetry and Remote Sensing (ISPRS) commission, were used to compare the performance of MHC with those of the 17 other publicized filtering methods. Results indicated that MHC with the average total error and average Cohen’s kappa coefficient of 4.11% and 86.27% performs better than all other filtering methods.

An Objective Classification of Saturn Cloud Features from Cassini ISS Images

NASA Technical Reports Server (NTRS)

Del Genio, Anthony D.; Barbara, John M.

2016-01-01

A k -means clustering algorithm is applied to Cassini Imaging Science Subsystem continuum and methane band images of Saturn's northern hemisphere to objectively classify regional albedo features and aid in their dynamical interpretation. The procedure is based on a technique applied previously to visible- infrared images of Earth. It provides a new perspective on giant planet cloud morphology and its relationship to the dynamics and a meteorological context for the analysis of other types of simultaneous Saturn observations. The method identifies 6 clusters that exhibit distinct morphology, vertical structure, and preferred latitudes of occurrence. These correspond to areas dominated by deep convective cells; low contrast areas, some including thinner and thicker clouds possibly associated with baroclinic instability; regions with possible isolated thin cirrus clouds; darker areas due to thinner low level clouds or clearer skies due to downwelling, or due to absorbing particles; and fields of relatively shallow cumulus clouds. The spatial associations among these cloud types suggest that dynamically, there are three distinct types of latitude bands on Saturn: deep convectively disturbed latitudes in cyclonic shear regions poleward of the eastward jets; convectively suppressed regions near and surrounding the westward jets; and baro-clinically unstable latitudes near eastward jet cores and in the anti-cyclonic regions equatorward of them. These are roughly analogous to some of the features of Earth's tropics, subtropics, and midlatitudes, respectively. This classification may be more useful for dynamics purposes than the traditional belt-zone partitioning. Temporal variations of feature contrast and cluster occurrence suggest that the upper tropospheric haze in the northern hemisphere may have thickened by 2014. The results suggest that routine use of clustering may be a worthwhile complement to many different types of planetary atmospheric data analysis.

Classification of cloud fields based on textural characteristics

NASA Technical Reports Server (NTRS)

Welch, R. M.; Sengupta, S. K.; Chen, D. W.

1987-01-01

The present study reexamines the applicability of texture-based features for automatic cloud classification using very high spatial resolution (57 m) Landsat multispectral scanner digital data. It is concluded that cloud classification can be accomplished using only a single visible channel.

Automatic 3D Extraction of Buildings, Vegetation and Roads from LIDAR Data

NASA Astrophysics Data System (ADS)

Bellakaout, A.; Cherkaoui, M.; Ettarid, M.; Touzani, A.

2016-06-01

Aerial topographic surveys using Light Detection and Ranging (LiDAR) technology collect dense and accurate information from the surface or terrain; it is becoming one of the important tools in the geosciences for studying objects and earth surface. Classification of Lidar data for extracting ground, vegetation, and buildings is a very important step needed in numerous applications such as 3D city modelling, extraction of different derived data for geographical information systems (GIS), mapping, navigation, etc... Regardless of what the scan data will be used for, an automatic process is greatly required to handle the large amount of data collected because the manual process is time consuming and very expensive. This paper is presenting an approach for automatic classification of aerial Lidar data into five groups of items: buildings, trees, roads, linear object and soil using single return Lidar and processing the point cloud without generating DEM. Topological relationship and height variation analysis is adopted to segment, preliminary, the entire point cloud preliminarily into upper and lower contours, uniform and non-uniform surface, non-uniform surfaces, linear objects, and others. This primary classification is used on the one hand to know the upper and lower part of each building in an urban scene, needed to model buildings façades; and on the other hand to extract point cloud of uniform surfaces which contain roofs, roads and ground used in the second phase of classification. A second algorithm is developed to segment the uniform surface into buildings roofs, roads and ground, the second phase of classification based on the topological relationship and height variation analysis, The proposed approach has been tested using two areas : the first is a housing complex and the second is a primary school. The proposed approach led to successful classification results of buildings, vegetation and road classes.

Cloud cover typing from environmental satellite imagery. Discriminating cloud structure with Fast Fourier Transforms (FFT)

NASA Technical Reports Server (NTRS)

Logan, T. L.; Huning, J. R.; Glackin, D. L.

1983-01-01

The use of two dimensional Fast Fourier Transforms (FFTs) subjected to pattern recognition technology for the identification and classification of low altitude stratus cloud structure from Geostationary Operational Environmental Satellite (GOES) imagery was examined. The development of a scene independent pattern recognition methodology, unconstrained by conventional cloud morphological classifications was emphasized. A technique for extracting cloud shape, direction, and size attributes from GOES visual imagery was developed. These attributes were combined with two statistical attributes (cloud mean brightness, cloud standard deviation), and interrogated using unsupervised clustering amd maximum likelihood classification techniques. Results indicate that: (1) the key cloud discrimination attributes are mean brightness, direction, shape, and minimum size; (2) cloud structure can be differentiated at given pixel scales; (3) cloud type may be identifiable at coarser scales; (4) there are positive indications of scene independence which would permit development of a cloud signature bank; (5) edge enhancement of GOES imagery does not appreciably improve cloud classification over the use of raw data; and (6) the GOES imagery must be apodized before generation of FFTs.

An Automated Cloud-edge Detection Algorithm Using Cloud Physics and Radar Data

NASA Technical Reports Server (NTRS)

Ward, Jennifer G.; Merceret, Francis J.; Grainger, Cedric A.

2003-01-01

An automated cloud edge detection algorithm was developed and extensively tested. The algorithm uses in-situ cloud physics data measured by a research aircraft coupled with ground-based weather radar measurements to determine whether the aircraft is in or out of cloud. Cloud edges are determined when the in/out state changes, subject to a hysteresis constraint. The hysteresis constraint prevents isolated transient cloud puffs or data dropouts from being identified as cloud boundaries. The algorithm was verified by detailed manual examination of the data set in comparison to the results from application of the automated algorithm.

Use of parallel computing in mass processing of laser data

NASA Astrophysics Data System (ADS)

Będkowski, J.; Bratuś, R.; Prochaska, M.; Rzonca, A.

2015-12-01

The first part of the paper includes a description of the rules used to generate the algorithm needed for the purpose of parallel computing and also discusses the origins of the idea of research on the use of graphics processors in large scale processing of laser scanning data. The next part of the paper includes the results of an efficiency assessment performed for an array of different processing options, all of which were substantially accelerated with parallel computing. The processing options were divided into the generation of orthophotos using point clouds, coloring of point clouds, transformations, and the generation of a regular grid, as well as advanced processes such as the detection of planes and edges, point cloud classification, and the analysis of data for the purpose of quality control. Most algorithms had to be formulated from scratch in the context of the requirements of parallel computing. A few of the algorithms were based on existing technology developed by the Dephos Software Company and then adapted to parallel computing in the course of this research study. Processing time was determined for each process employed for a typical quantity of data processed, which helped confirm the high efficiency of the solutions proposed and the applicability of parallel computing to the processing of laser scanning data. The high efficiency of parallel computing yields new opportunities in the creation and organization of processing methods for laser scanning data.

Automated cloud classification with a fuzzy logic expert system

NASA Technical Reports Server (NTRS)

Tovinkere, Vasanth; Baum, Bryan A.

1993-01-01

An unresolved problem in current cloud retrieval algorithms concerns the analysis of scenes containing overlapping cloud layers. Cloud parameterizations are very important both in global climate models and in studies of the Earth's radiation budget. Most cloud retrieval schemes, such as the bispectral method used by the International Satellite Cloud Climatology Project (ISCCP), have no way of determining whether overlapping cloud layers exist in any group of satellite pixels. One promising method uses fuzzy logic to determine whether mixed cloud and/or surface types exist within a group of pixels, such as cirrus, land, and water, or cirrus and stratus. When two or more class types are present, fuzzy logic uses membership values to assign the group of pixels partially to the different class types. The strength of fuzzy logic lies in its ability to work with patterns that may include more than one class, facilitating greater information extraction from satellite radiometric data. The development of the fuzzy logic rule-based expert system involves training the fuzzy classifier with spectral and textural features calculated from accurately labeled 32x32 regions of Advanced Very High Resolution Radiometer (AVHRR) 1.1-km data. The spectral data consists of AVHRR channels 1 (0.55-0.68 mu m), 2 (0.725-1.1 mu m), 3 (3.55-3.93 mu m), 4 (10.5-11.5 mu m), and 5 (11.5-12.5 mu m), which include visible, near-infrared, and infrared window regions. The textural features are based on the gray level difference vector (GLDV) method. A sophisticated new interactive visual image Classification System (IVICS) is used to label samples chosen from scenes collected during the FIRE IFO II. The training samples are chosen from predefined classes, chosen to be ocean, land, unbroken stratiform, broken stratiform, and cirrus. The November 28, 1991 NOAA overpasses contain complex multilevel cloud situations ideal for training and validating the fuzzy logic expert system.

What's the Point of a Raster ? Advantages of 3D Point Cloud Processing over Raster Based Methods for Accurate Geomorphic Analysis of High Resolution Topography.

NASA Astrophysics Data System (ADS)

Lague, D.

2014-12-01

High Resolution Topographic (HRT) datasets are predominantly stored and analyzed as 2D raster grids of elevations (i.e., Digital Elevation Models). Raster grid processing is common in GIS software and benefits from a large library of fast algorithms dedicated to geometrical analysis, drainage network computation and topographic change measurement. Yet, all instruments or methods currently generating HRT datasets (e.g., ALS, TLS, SFM, stereo satellite imagery) output natively 3D unstructured point clouds that are (i) non-regularly sampled, (ii) incomplete (e.g., submerged parts of river channels are rarely measured), and (iii) include 3D elements (e.g., vegetation, vertical features such as river banks or cliffs) that cannot be accurately described in a DEM. Interpolating the raw point cloud onto a 2D grid generally results in a loss of position accuracy, spatial resolution and in more or less controlled interpolation. Here I demonstrate how studying earth surface topography and processes directly on native 3D point cloud datasets offers several advantages over raster based methods: point cloud methods preserve the accuracy of the original data, can better handle the evaluation of uncertainty associated to topographic change measurements and are more suitable to study vegetation characteristics and steep features of the landscape. In this presentation, I will illustrate and compare Point Cloud based and Raster based workflows with various examples involving ALS, TLS and SFM for the analysis of bank erosion processes in bedrock and alluvial rivers, rockfall statistics (including rockfall volume estimate directly from point clouds) and the interaction of vegetation/hydraulics and sedimentation in salt marshes. These workflows use 2 recently published algorithms for point cloud classification (CANUPO) and point cloud comparison (M3C2) now implemented in the open source software CloudCompare.

Cloud detection algorithm comparison and validation for operational Landsat data products

USGS Publications Warehouse

Foga, Steven Curtis; Scaramuzza, Pat; Guo, Song; Zhu, Zhe; Dilley, Ronald; Beckmann, Tim; Schmidt, Gail L.; Dwyer, John L.; Hughes, MJ; Laue, Brady

2017-01-01

Clouds are a pervasive and unavoidable issue in satellite-borne optical imagery. Accurate, well-documented, and automated cloud detection algorithms are necessary to effectively leverage large collections of remotely sensed data. The Landsat project is uniquely suited for comparative validation of cloud assessment algorithms because the modular architecture of the Landsat ground system allows for quick evaluation of new code, and because Landsat has the most comprehensive manual truth masks of any current satellite data archive. Currently, the Landsat Level-1 Product Generation System (LPGS) uses separate algorithms for determining clouds, cirrus clouds, and snow and/or ice probability on a per-pixel basis. With more bands onboard the Landsat 8 Operational Land Imager (OLI)/Thermal Infrared Sensor (TIRS) satellite, and a greater number of cloud masking algorithms, the U.S. Geological Survey (USGS) is replacing the current cloud masking workflow with a more robust algorithm that is capable of working across multiple Landsat sensors with minimal modification. Because of the inherent error from stray light and intermittent data availability of TIRS, these algorithms need to operate both with and without thermal data. In this study, we created a workflow to evaluate cloud and cloud shadow masking algorithms using cloud validation masks manually derived from both Landsat 7 Enhanced Thematic Mapper Plus (ETM +) and Landsat 8 OLI/TIRS data. We created a new validation dataset consisting of 96 Landsat 8 scenes, representing different biomes and proportions of cloud cover. We evaluated algorithm performance by overall accuracy, omission error, and commission error for both cloud and cloud shadow. We found that CFMask, C code based on the Function of Mask (Fmask) algorithm, and its confidence bands have the best overall accuracy among the many algorithms tested using our validation data. The Artificial Thermal-Automated Cloud Cover Algorithm (AT-ACCA) is the most accurate nonthermal-based algorithm. We give preference to CFMask for operational cloud and cloud shadow detection, as it is derived from a priori knowledge of physical phenomena and is operable without geographic restriction, making it useful for current and future land imaging missions without having to be retrained in a machine-learning environment.

QoS-aware health monitoring system using cloud-based WBANs.

PubMed

Almashaqbeh, Ghada; Hayajneh, Thaier; Vasilakos, Athanasios V; Mohd, Bassam J

2014-10-01

Wireless Body Area Networks (WBANs) are amongst the best options for remote health monitoring. However, as standalone systems WBANs have many limitations due to the large amount of processed data, mobility of monitored users, and the network coverage area. Integrating WBANs with cloud computing provides effective solutions to these problems and promotes the performance of WBANs based systems. Accordingly, in this paper we propose a cloud-based real-time remote health monitoring system for tracking the health status of non-hospitalized patients while practicing their daily activities. Compared with existing cloud-based WBAN frameworks, we divide the cloud into local one, that includes the monitored users and local medical staff, and a global one that includes the outer world. The performance of the proposed framework is optimized by reducing congestion, interference, and data delivery delay while supporting users' mobility. Several novel techniques and algorithms are proposed to accomplish our objective. First, the concept of data classification and aggregation is utilized to avoid clogging the network with unnecessary data traffic. Second, a dynamic channel assignment policy is developed to distribute the WBANs associated with the users on the available frequency channels to manage interference. Third, a delay-aware routing metric is proposed to be used by the local cloud in its multi-hop communication to speed up the reporting process of the health-related data. Fourth, the delay-aware metric is further utilized by the association protocols used by the WBANs to connect with the local cloud. Finally, the system with all the proposed techniques and algorithms is evaluated using extensive ns-2 simulations. The simulation results show superior performance of the proposed architecture in optimizing the end-to-end delay, handling the increased interference levels, maximizing the network capacity, and tracking user's mobility.

Seasonal Surface Spectral Emissivity Derived from Terra MODIS Data

NASA Technical Reports Server (NTRS)

Sun-Mack, Sunny; Chen, Yan; Minnis, Patrick; Young, DavidF.; Smith, William J., Jr.

2004-01-01

The CERES (Clouds and the Earth's Radiant Energy System) Project is measuring broadband shortwave and longwave radiances and deriving cloud properties form various images to produce a combined global radiation and cloud property data set. In this paper, simultaneous data from Terra MODIS (Moderate Resolution Imaging Spectroradiometer) taken at 3.7, 8.5, 11.0, and 12.0 m are used to derive the skin temperature and the surface emissivities at the same wavelengths. The methodology uses separate measurements of clear sky temperature in each channel determined by scene classification during the daytime and at night. The relationships between the various channels at night are used during the day when solar reflectance affects the 3.7- m radiances. A set of simultaneous equations is then solved to derive the emissivities. Global monthly emissivity maps are derived from Terra MODIS data while numerical weather analyses provide soundings for correcting the observed radiances for atmospheric absorption. These maps are used by CERES and other cloud retrieval algorithms.

Optimizing selection of training and auxiliary data for operational land cover classification for the LCMAP initiative

NASA Astrophysics Data System (ADS)

Zhu, Zhe; Gallant, Alisa L.; Woodcock, Curtis E.; Pengra, Bruce; Olofsson, Pontus; Loveland, Thomas R.; Jin, Suming; Dahal, Devendra; Yang, Limin; Auch, Roger F.

2016-12-01

The U.S. Geological Survey's Land Change Monitoring, Assessment, and Projection (LCMAP) initiative is a new end-to-end capability to continuously track and characterize changes in land cover, use, and condition to better support research and applications relevant to resource management and environmental change. Among the LCMAP product suite are annual land cover maps that will be available to the public. This paper describes an approach to optimize the selection of training and auxiliary data for deriving the thematic land cover maps based on all available clear observations from Landsats 4-8. Training data were selected from map products of the U.S. Geological Survey's Land Cover Trends project. The Random Forest classifier was applied for different classification scenarios based on the Continuous Change Detection and Classification (CCDC) algorithm. We found that extracting training data proportionally to the occurrence of land cover classes was superior to an equal distribution of training data per class, and suggest using a total of 20,000 training pixels to classify an area about the size of a Landsat scene. The problem of unbalanced training data was alleviated by extracting a minimum of 600 training pixels and a maximum of 8000 training pixels per class. We additionally explored removing outliers contained within the training data based on their spectral and spatial criteria, but observed no significant improvement in classification results. We also tested the importance of different types of auxiliary data that were available for the conterminous United States, including: (a) five variables used by the National Land Cover Database, (b) three variables from the cloud screening "Function of mask" (Fmask) statistics, and (c) two variables from the change detection results of CCDC. We found that auxiliary variables such as a Digital Elevation Model and its derivatives (aspect, position index, and slope), potential wetland index, water probability, snow probability, and cloud probability improved the accuracy of land cover classification. Compared to the original strategy of the CCDC algorithm (500 pixels per class), the use of the optimal strategy improved the classification accuracies substantially (15-percentage point increase in overall accuracy and 4-percentage point increase in minimum accuracy).

Ground-based cloud classification by learning stable local binary patterns

NASA Astrophysics Data System (ADS)

Wang, Yu; Shi, Cunzhao; Wang, Chunheng; Xiao, Baihua

2018-07-01

Feature selection and extraction is the first step in implementing pattern classification. The same is true for ground-based cloud classification. Histogram features based on local binary patterns (LBPs) are widely used to classify texture images. However, the conventional uniform LBP approach cannot capture all the dominant patterns in cloud texture images, thereby resulting in low classification performance. In this study, a robust feature extraction method by learning stable LBPs is proposed based on the averaged ranks of the occurrence frequencies of all rotation invariant patterns defined in the LBPs of cloud images. The proposed method is validated with a ground-based cloud classification database comprising five cloud types. Experimental results demonstrate that the proposed method achieves significantly higher classification accuracy than the uniform LBP, local texture patterns (LTP), dominant LBP (DLBP), completed LBP (CLTP) and salient LBP (SaLBP) methods in this cloud image database and under different noise conditions. And the performance of the proposed method is comparable with that of the popular deep convolutional neural network (DCNN) method, but with less computation complexity. Furthermore, the proposed method also achieves superior performance on an independent test data set.

Experimental evaluation of ALS point cloud ground extraction over different land cover in the Malopolska Province

NASA Astrophysics Data System (ADS)

Korzeniowska, Karolina; Mandlburger, Gottfried; Klimczyk, Agata

2013-04-01

The paper presents an evaluation of different terrain point extraction algorithms for Airborne Laser Scanning (ALS) point clouds. The research area covers eight test sites in the Małopolska Province (Poland) with varying point density between 3-15points/m² and surface as well as land cover characteristics. In this paper the existing implementations of algorithms were considered. Approaches based on mathematical morphology, progressive densification, robust surface interpolation and segmentation were compared. From the group of morphological filters, the Progressive Morphological Filter (PMF) proposed by Zhang K. et al. (2003) in LIS software was evaluated. From the progressive densification filter methods developed by Axelsson P. (2000) the Martin Isenburg's implementation in LAStools software (LAStools, 2012) was chosen. The third group of methods are surface-based filters. In this study, we used the hierarchic robust interpolation approach by Kraus K., Pfeifer N. (1998) as implemented in SCOP++ (Trimble, 2012). The fourth group of methods works on segmentation. From this filtering concept the segmentation algorithm available in LIS was tested (Wichmann V., 2012). The main aim in executing the automatic classification for ground extraction was operating in default mode or with default parameters which were selected by the developers of the algorithms. It was assumed that the default settings were equivalent to the parameters on which the best results can be achieved. In case it was not possible to apply an algorithm in default mode, a combination of the available and most crucial parameters for ground extraction were selected. As a result of these analyses, several output LAS files with different ground classification were achieved. The results were described on the basis of qualitative and quantitative analyses, both being in a formal description. The classification differences were verified on point cloud data. Qualitative verification of ground extraction was made on the basis of a visual inspection of the results (Sithole G., Vosselman G., 2004; Meng X. et al., 2010). The results of these analyses were described as a graph using weighted assumption. The quantitative analyses were evaluated on a basis of Type I, Type II and Total errors (Sithole G., Vosselman G., 2003). The achieved results show that the analysed algorithms yield different classification accuracies depending on the landscape and land cover. The simplest terrain for ground extraction was flat rural area with sparse vegetation. The most difficult were mountainous areas with very dense vegetation where only a few ground points were available. Generally the LAStools algorithm gives good results in every type of terrain, but the ground surface is too smooth. The LIS Progressive Morphological Filter algorithm gives good results in forested flat and low slope areas. The surface-based algorithm from SCOP++ gives good results in mountainous areas - both forested and built-up because it better preserves steep slopes, sharp ridges and breaklines, but sometimes it fails to remove off-terrain objects from the ground class. The segmentation-based algorithm in LIS gives quite good results in built-up flat areas, but in forested areas it does not work well. Bibliography: Axelsson, P., 2000. DEM generation from laser scanner data using adaptive TIN models. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XXXIII (Pt. B4/1), 110- 117 Kraus, K., Pfeifer, N., 1998. Determination of terrain models in wooded areas with airborne laser scanner data. ISPRS Journal of Photogrammetry & Remote Sensing 53 (4), 193-203 LAStools website http://www.cs.unc.edu/~isenburg/lastools/ (verified in September 2012) Meng, X., Currit, N., Zhao, K., 2010. Ground Filtering Algorithms for Airborne LiDAR Data: A Review of Critical Issues. Remote Sensing 2, 833-860 Sithole, G., Vosselman, G., 2003. Report: ISPRS Comparison of Filters. Commission III, Working Group 3. Department of Geodesy, Faculty of Civil Engineering and Geosciences, Delft University of technology, The Netherlands Sithole, G., Vosselman, G., 2004. Experimental comparison of filter algorithms for bare-Earth extraction form airborne laser scanning point clouds. ISPRS Journal of Photogrammetry & Remote Sensing 59, 85-101 Trimble, 2012 http://www.trimble.com/geospatial/aerial-software.aspx (verified in November 2012) Wichmann, V., 2012. LIS Command Reference, LASERDATA GmbH, 1-231 Zhang, K., Chen, S.-C., Whitman, D., Shyu, M.-L., Yan, J., Zhang, C., 2003. A progressive morphological filter for removing non-ground measurements from airborne LIDAR data. IEEE Transactions on Geoscience and Remote Sensing, 41(4), 872-882

Phenotype Instance Verification and Evaluation Tool (PIVET): A Scaled Phenotype Evidence Generation Framework Using Web-Based Medical Literature.

PubMed

Henderson, Jette; Ke, Junyuan; Ho, Joyce C; Ghosh, Joydeep; Wallace, Byron C

2018-05-04

Researchers are developing methods to automatically extract clinically relevant and useful patient characteristics from raw healthcare datasets. These characteristics, often capturing essential properties of patients with common medical conditions, are called computational phenotypes. Being generated by automated or semiautomated, data-driven methods, such potential phenotypes need to be validated as clinically meaningful (or not) before they are acceptable for use in decision making. The objective of this study was to present Phenotype Instance Verification and Evaluation Tool (PIVET), a framework that uses co-occurrence analysis on an online corpus of publically available medical journal articles to build clinical relevance evidence sets for user-supplied phenotypes. PIVET adopts a conceptual framework similar to the pioneering prototype tool PheKnow-Cloud that was developed for the phenotype validation task. PIVET completely refactors each part of the PheKnow-Cloud pipeline to deliver vast improvements in speed without sacrificing the quality of the insights PheKnow-Cloud achieved. PIVET leverages indexing in NoSQL databases to efficiently generate evidence sets. Specifically, PIVET uses a succinct representation of the phenotypes that corresponds to the index on the corpus database and an optimized co-occurrence algorithm inspired by the Aho-Corasick algorithm. We compare PIVET's phenotype representation with PheKnow-Cloud's by using PheKnow-Cloud's experimental setup. In PIVET's framework, we also introduce a statistical model trained on domain expert-verified phenotypes to automatically classify phenotypes as clinically relevant or not. Additionally, we show how the classification model can be used to examine user-supplied phenotypes in an online, rather than batch, manner. PIVET maintains the discriminative power of PheKnow-Cloud in terms of identifying clinically relevant phenotypes for the same corpus with which PheKnow-Cloud was originally developed, but PIVET's analysis is an order of magnitude faster than that of PheKnow-Cloud. Not only is PIVET much faster, it can be scaled to a larger corpus and still retain speed. We evaluated multiple classification models on top of the PIVET framework and found ridge regression to perform best, realizing an average F1 score of 0.91 when predicting clinically relevant phenotypes. Our study shows that PIVET improves on the most notable existing computational tool for phenotype validation in terms of speed and automation and is comparable in terms of accuracy. ©Jette Henderson, Junyuan Ke, Joyce C Ho, Joydeep Ghosh, Byron C Wallace. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 04.05.2018.

An Iterative Closest Points Algorithm for Registration of 3D Laser Scanner Point Clouds with Geometric Features.

PubMed

He, Ying; Liang, Bin; Yang, Jun; Li, Shunzhi; He, Jin

2017-08-11

The Iterative Closest Points (ICP) algorithm is the mainstream algorithm used in the process of accurate registration of 3D point cloud data. The algorithm requires a proper initial value and the approximate registration of two point clouds to prevent the algorithm from falling into local extremes, but in the actual point cloud matching process, it is difficult to ensure compliance with this requirement. In this paper, we proposed the ICP algorithm based on point cloud features (GF-ICP). This method uses the geometrical features of the point cloud to be registered, such as curvature, surface normal and point cloud density, to search for the correspondence relationships between two point clouds and introduces the geometric features into the error function to realize the accurate registration of two point clouds. The experimental results showed that the algorithm can improve the convergence speed and the interval of convergence without setting a proper initial value.

An Iterative Closest Points Algorithm for Registration of 3D Laser Scanner Point Clouds with Geometric Features

PubMed Central

Liang, Bin; Yang, Jun; Li, Shunzhi; He, Jin

2017-01-01

The Iterative Closest Points (ICP) algorithm is the mainstream algorithm used in the process of accurate registration of 3D point cloud data. The algorithm requires a proper initial value and the approximate registration of two point clouds to prevent the algorithm from falling into local extremes, but in the actual point cloud matching process, it is difficult to ensure compliance with this requirement. In this paper, we proposed the ICP algorithm based on point cloud features (GF-ICP). This method uses the geometrical features of the point cloud to be registered, such as curvature, surface normal and point cloud density, to search for the correspondence relationships between two point clouds and introduces the geometric features into the error function to realize the accurate registration of two point clouds. The experimental results showed that the algorithm can improve the convergence speed and the interval of convergence without setting a proper initial value. PMID:28800096

Multilayer Cloud Detection with the MODIS Near-Infrared Water Vapor Absorption Band

NASA Technical Reports Server (NTRS)

Wind, Galina; Platnick, Steven; King, Michael D.; Hubanks, Paul A,; Pavolonis, Michael J.; Heidinger, Andrew K.; Yang, Ping; Baum, Bryan A.

2009-01-01

Data Collection 5 processing for the Moderate Resolution Imaging Spectroradiometer (MODIS) onboard the NASA Earth Observing System EOS Terra and Aqua spacecraft includes an algorithm for detecting multilayered clouds in daytime. The main objective of this algorithm is to detect multilayered cloud scenes, specifically optically thin ice cloud overlying a lower-level water cloud, that presents difficulties for retrieving cloud effective radius using single layer plane-parallel cloud models. The algorithm uses the MODIS 0.94 micron water vapor band along with CO2 bands to obtain two above-cloud precipitable water retrievals, the difference of which, in conjunction with additional tests, provides a map of where multilayered clouds might potentially exist. The presence of a multilayered cloud results in a large difference in retrievals of above-cloud properties between the CO2 and the 0.94 micron methods. In this paper the MODIS multilayered cloud algorithm is described, results of using the algorithm over example scenes are shown, and global statistics for multilayered clouds as observed by MODIS are discussed. A theoretical study of the algorithm behavior for simulated multilayered clouds is also given. Results are compared to two other comparable passive imager methods. A set of standard cloudy atmospheric profiles developed during the course of this investigation is also presented. The results lead to the conclusion that the MODIS multilayer cloud detection algorithm has some skill in identifying multilayered clouds with different thermodynamic phases

The Weather Forecast Using Data Mining Research Based on Cloud Computing.

NASA Astrophysics Data System (ADS)

Wang, ZhanJie; Mazharul Mujib, A. B. M.

2017-10-01

Weather forecasting has been an important application in meteorology and one of the most scientifically and technologically challenging problem around the world. In my study, we have analyzed the use of data mining techniques in forecasting weather. This paper proposes a modern method to develop a service oriented architecture for the weather information systems which forecast weather using these data mining techniques. This can be carried out by using Artificial Neural Network and Decision tree Algorithms and meteorological data collected in Specific time. Algorithm has presented the best results to generate classification rules for the mean weather variables. The results showed that these data mining techniques can be enough for weather forecasting.

Classification by Using Multispectral Point Cloud Data

NASA Astrophysics Data System (ADS)

Liao, C. T.; Huang, H. H.

2012-07-01

Remote sensing images are generally recorded in two-dimensional format containing multispectral information. Also, the semantic information is clearly visualized, which ground features can be better recognized and classified via supervised or unsupervised classification methods easily. Nevertheless, the shortcomings of multispectral images are highly depending on light conditions, and classification results lack of three-dimensional semantic information. On the other hand, LiDAR has become a main technology for acquiring high accuracy point cloud data. The advantages of LiDAR are high data acquisition rate, independent of light conditions and can directly produce three-dimensional coordinates. However, comparing with multispectral images, the disadvantage is multispectral information shortage, which remains a challenge in ground feature classification through massive point cloud data. Consequently, by combining the advantages of both LiDAR and multispectral images, point cloud data with three-dimensional coordinates and multispectral information can produce a integrate solution for point cloud classification. Therefore, this research acquires visible light and near infrared images, via close range photogrammetry, by matching images automatically through free online service for multispectral point cloud generation. Then, one can use three-dimensional affine coordinate transformation to compare the data increment. At last, the given threshold of height and color information is set as threshold in classification.

Assessment of the NPOESS/VIIRS Nighttime Infrared Cloud Optical Properties Algorithms

NASA Astrophysics Data System (ADS)

Wong, E.; Ou, S. C.

2008-12-01

In this paper we will describe two NPOESS VIIRS IR algorithms used to retrieve microphysical properties for water and ice clouds during nighttime conditions. Both algorithms employ four VIIRS IR channels: M12 (3.7 μm), M14 (8.55 μm), M15 (10.7 μm) and M16 (12 μm). The physical basis for the two algorithms is similar in that while the Cloud Top Temperature (CTT) is derived from M14 and M16 for ice clouds the Cloud Optical Thickness (COT) and Cloud Effective Particle Size (CEPS) are derived from M12 and M15. The two algorithms depart in the different radiative transfer parameterization equations used for ice and water clouds. Both the VIIRS nighttime IR algorithms and the CERES split-window method employ the 3.7 μm and 10.7 μm bands for cloud optical properties retrievals, apparently based on similar physical principles but with different implementations. It is reasonable to expect that the VIIRS and CERES IR algorithms produce comparable performance and similar limitations. To demonstrate the VIIRS nighttime IR algorithm performance, we will select a number of test cases using NASA MODIS L1b radiance products as proxy input data for VIIRS. The VIIRS retrieved COT and CEPS will then be compared to cloud products available from the MODIS, NASA CALIPSO, CloudSat and CERES sensors. For the MODIS product, the nighttime cloud emissivity will serve as an indirect comparison to VIIRS COT. For the CALIPSO and CloudSat products, the layered COT will be used for direct comparison. Finally, the CERES products will provide direct comparison with COT as well as CEPS. This study can only provide a qualitative assessment of the VIIRS IR algorithms due to the large uncertainties in these cloud products.

Remote Sensing of Cloud Properties using Ground-based Measurements of Zenith Radiance

NASA Technical Reports Server (NTRS)

Chiu, J. Christine; Marshak, Alexander; Knyazikhin, Yuri; Wiscombe, Warren J.; Barker, Howard W.; Barnard, James C.; Luo, Yi

2006-01-01

An extensive verification of cloud property retrievals has been conducted for two algorithms using zenith radiances measured by the Atmospheric Radiation Measurement (ARM) Program ground-based passive two-channel (673 and 870 nm) Narrow Field-Of-View Radiometer. The underlying principle of these algorithms is that clouds have nearly identical optical properties at these wavelengths, but corresponding spectral surface reflectances (for vegetated surfaces) differ significantly. The first algorithm, the RED vs. NIR, works for a fully three-dimensional cloud situation. It retrieves not only cloud optical depth, but also an effective radiative cloud fraction. Importantly, due to one-second time resolution of radiance measurements, we are able, for the first time, to capture detailed changes in cloud structure at the natural time scale of cloud evolution. The cloud optical depths tau retrieved by this algorithm are comparable to those inferred from both downward fluxes in overcast situations and microwave brightness temperatures for broken clouds. Moreover, it can retrieve tau for thin patchy clouds, where flux and microwave observations fail to detect them. The second algorithm, referred to as COUPLED, couples zenith radiances with simultaneous fluxes to infer 2. In general, the COUPLED and RED vs. NIR algorithms retrieve consistent values of tau. However, the COUPLED algorithm is more sensitive to the accuracies of measured radiance, flux, and surface reflectance than the RED vs. NIR algorithm. This is especially true for thick overcast clouds where it may substantially overestimate z.

Objected-oriented remote sensing image classification method based on geographic ontology model

NASA Astrophysics Data System (ADS)

Chu, Z.; Liu, Z. J.; Gu, H. Y.

2016-11-01

Nowadays, with the development of high resolution remote sensing image and the wide application of laser point cloud data, proceeding objected-oriented remote sensing classification based on the characteristic knowledge of multi-source spatial data has been an important trend on the field of remote sensing image classification, which gradually replaced the traditional method through improving algorithm to optimize image classification results. For this purpose, the paper puts forward a remote sensing image classification method that uses the he characteristic knowledge of multi-source spatial data to build the geographic ontology semantic network model, and carries out the objected-oriented classification experiment to implement urban features classification, the experiment uses protégé software which is developed by Stanford University in the United States, and intelligent image analysis software—eCognition software as the experiment platform, uses hyperspectral image and Lidar data that is obtained through flight in DaFeng City of JiangSu as the main data source, first of all, the experiment uses hyperspectral image to obtain feature knowledge of remote sensing image and related special index, the second, the experiment uses Lidar data to generate nDSM(Normalized DSM, Normalized Digital Surface Model),obtaining elevation information, the last, the experiment bases image feature knowledge, special index and elevation information to build the geographic ontology semantic network model that implement urban features classification, the experiment results show that, this method is significantly higher than the traditional classification algorithm on classification accuracy, especially it performs more evidently on the respect of building classification. The method not only considers the advantage of multi-source spatial data, for example, remote sensing image, Lidar data and so on, but also realizes multi-source spatial data knowledge integration and application of the knowledge to the field of remote sensing image classification, which provides an effective way for objected-oriented remote sensing image classification in the future.

Probability Density Functions for the CALIPSO Lidar Version 4 Cloud-Aerosol Discrimination (CAD) Algorithm

NASA Astrophysics Data System (ADS)

Liu, Z.; Kar, J.; Zeng, S.; Tackett, J. L.; Vaughan, M.; Trepte, C. R.; Omar, A. H.; Hu, Y.; Winker, D. M.

2017-12-01

In the CALIPSO retrieval algorithm, detection layers in the lidar measurements is followed by their classification as a "cloud" or "aerosol" using 5-dimensional probability density functions (PDFs). The five dimensions are the mean attenuated backscatter at 532 nm, the layer integrated total attenuated color ratio, the mid-layer altitude, integrated volume depolarization ratio and latitude. The new version 4 (V4) level 2 (L2) data products, released in November 2016, are the first major revision to the L2 product suite since May 2010. Significant calibration changes in the V4 level 1 data necessitated substantial revisions to the V4 L2 CAD algorithm. Accordingly, a new set of PDFs was generated to derive the V4 L2 data products. The V4 CAD algorithm is now applied to layers detected in the stratosphere, where volcanic layers and occasional cloud and smoke layers are observed. Previously, these layers were designated as `stratospheric', and not further classified. The V4 CAD algorithm is also applied to all layers detected at single shot (333 m) resolution. In prior data releases, single shot detections were uniformly classified as clouds. The CAD PDFs used in the earlier releases were generated using a full year (2008) of CALIPSO measurements. Because the CAD algorithm was not applied to stratospheric features, the properties of these layers were not incorporated into the PDFs. When building the V4 PDFs, the 2008 data were augmented with additional data from June 2011, and all stratospheric features were included. The Nabro and Puyehue-Cordon volcanos erupted in June 2011, and volcanic aerosol layers were observed in the upper troposphere and lower stratosphere in both the northern and southern hemispheres. The June 2011 data thus provides the stratospheric aerosol properties needed for comprehensive PDF generation. In contrast to earlier versions of the PDFs, which were generated based solely on observed distributions, construction of the V4 PDFs considered the typical optical and physical properties of feature subtypes, and thus provide a more comprehensive physical basis for discrimination. As a result of the changes made, the V4 CAD provides better performance and more reliable confidence levels. We describe the generation of V4 PDFs and present characterization and performance of the new CAD algorithm.

Classification of Arctic, midlatitude and tropical clouds in the mixed-phase temperature regime

NASA Astrophysics Data System (ADS)

Costa, Anja; Meyer, Jessica; Afchine, Armin; Luebke, Anna; Günther, Gebhard; Dorsey, James R.; Gallagher, Martin W.; Ehrlich, Andre; Wendisch, Manfred; Baumgardner, Darrel; Wex, Heike; Krämer, Martina

2017-10-01

The degree of glaciation of mixed-phase clouds constitutes one of the largest uncertainties in climate prediction. In order to better understand cloud glaciation, cloud spectrometer observations are presented in this paper, which were made in the mixed-phase temperature regime between 0 and -38 °C (273 to 235 K), where cloud particles can either be frozen or liquid. The extensive data set covers four airborne field campaigns providing a total of 139 000 1 Hz data points (38.6 h within clouds) over Arctic, midlatitude and tropical regions. We develop algorithms, combining the information on number concentration, size and asphericity of the observed cloud particles to classify four cloud types: liquid clouds, clouds in which liquid droplets and ice crystals coexist, fully glaciated clouds after the Wegener-Bergeron-Findeisen process and clouds where secondary ice formation occurred. We quantify the occurrence of these cloud groups depending on the geographical region and temperature and find that liquid clouds dominate our measurements during the Arctic spring, while clouds dominated by the Wegener-Bergeron-Findeisen process are most common in midlatitude spring. The coexistence of liquid water and ice crystals is found over the whole mixed-phase temperature range in tropical convective towers in the dry season. Secondary ice is found at midlatitudes at -5 to -10 °C (268 to 263 K) and at higher altitudes, i.e. lower temperatures in the tropics. The distribution of the cloud types with decreasing temperature is shown to be consistent with the theory of evolution of mixed-phase clouds. With this study, we aim to contribute to a large statistical database on cloud types in the mixed-phase temperature regime.

Refinements to HIRS CO2 Slicing Algorithm with Results Compared to CALIOP and MODIS

NASA Astrophysics Data System (ADS)

Frey, R.; Menzel, P.

2012-12-01

This poster reports on the refinement of a cloud top property algorithm using High-resolution Infrared Radiation Sounder (HIRS) measurements. The HIRS sensor has been flown on fifteen satellites from TIROS-N through NOAA-19 and MetOp-A forming a continuous 30 year cloud data record. Cloud Top Pressure and effective emissivity (cloud fraction multiplied by cloud emissivity) are derived using the 15 μm spectral bands in the CO2 absorption band, implementing the CO2 slicing technique which is strong for high semi-transparent clouds but weak for low clouds with little thermal contrast from clear skies. We report on algorithm adjustments suggested from MODIS cloud record validations and the inclusion of collocated AVHRR cloud fraction data from the PATMOS-x algorithm. Reprocessing results for 2008 are shown using NOAA-18 HIRS and collocated CALIOP data for validation, as well as comparisons to MODIS monthly mean values. Adjustments to the cloud algorithm include (a) using CO2 slicing for all ice and mixed phase clouds and infrared window determinations for all water clouds, (b) determining the cloud top pressure from the most opaque CO2 spectral band pair seeing the cloud, (c) reducing the cloud detection threshold for the CO2 slicing algorithm to include conditions of smaller radiance differences that are often due to thin ice clouds, and (d) identifying stratospheric clouds when an opaque band is warmer than a less opaque band.

Automatic epileptic seizure detection in EEGs using MF-DFA, SVM based on cloud computing.

PubMed

Zhang, Zhongnan; Wen, Tingxi; Huang, Wei; Wang, Meihong; Li, Chunfeng

2017-01-01

Epilepsy is a chronic disease with transient brain dysfunction that results from the sudden abnormal discharge of neurons in the brain. Since electroencephalogram (EEG) is a harmless and noninvasive detection method, it plays an important role in the detection of neurological diseases. However, the process of analyzing EEG to detect neurological diseases is often difficult because the brain electrical signals are random, non-stationary and nonlinear. In order to overcome such difficulty, this study aims to develop a new computer-aided scheme for automatic epileptic seizure detection in EEGs based on multi-fractal detrended fluctuation analysis (MF-DFA) and support vector machine (SVM). New scheme first extracts features from EEG by MF-DFA during the first stage. Then, the scheme applies a genetic algorithm (GA) to calculate parameters used in SVM and classify the training data according to the selected features using SVM. Finally, the trained SVM classifier is exploited to detect neurological diseases. The algorithm utilizes MLlib from library of SPARK and runs on cloud platform. Applying to a public dataset for experiment, the study results show that the new feature extraction method and scheme can detect signals with less features and the accuracy of the classification reached up to 99%. MF-DFA is a promising approach to extract features for analyzing EEG, because of its simple algorithm procedure and less parameters. The features obtained by MF-DFA can represent samples as well as traditional wavelet transform and Lyapunov exponents. GA can always find useful parameters for SVM with enough execution time. The results illustrate that the classification model can achieve comparable accuracy, which means that it is effective in epileptic seizure detection.

Scalable Machine Learning for Massive Astronomical Datasets

NASA Astrophysics Data System (ADS)

Ball, Nicholas M.; Gray, A.

2014-04-01

We present the ability to perform data mining and machine learning operations on a catalog of half a billion astronomical objects. This is the result of the combination of robust, highly accurate machine learning algorithms with linear scalability that renders the applications of these algorithms to massive astronomical data tractable. We demonstrate the core algorithms kernel density estimation, K-means clustering, linear regression, nearest neighbors, random forest and gradient-boosted decision tree, singular value decomposition, support vector machine, and two-point correlation function. Each of these is relevant for astronomical applications such as finding novel astrophysical objects, characterizing artifacts in data, object classification (including for rare objects), object distances, finding the important features describing objects, density estimation of distributions, probabilistic quantities, and exploring the unknown structure of new data. The software, Skytree Server, runs on any UNIX-based machine, a virtual machine, or cloud-based and distributed systems including Hadoop. We have integrated it on the cloud computing system of the Canadian Astronomical Data Centre, the Canadian Advanced Network for Astronomical Research (CANFAR), creating the world's first cloud computing data mining system for astronomy. We demonstrate results showing the scaling of each of our major algorithms on large astronomical datasets, including the full 470,992,970 objects of the 2 Micron All-Sky Survey (2MASS) Point Source Catalog. We demonstrate the ability to find outliers in the full 2MASS dataset utilizing multiple methods, e.g., nearest neighbors. This is likely of particular interest to the radio astronomy community given, for example, that survey projects contain groups dedicated to this topic. 2MASS is used as a proof-of-concept dataset due to its convenience and availability. These results are of interest to any astronomical project with large and/or complex datasets that wishes to extract the full scientific value from its data.

Scalable Machine Learning for Massive Astronomical Datasets

NASA Astrophysics Data System (ADS)

Ball, Nicholas M.; Astronomy Data Centre, Canadian

2014-01-01

We present the ability to perform data mining and machine learning operations on a catalog of half a billion astronomical objects. This is the result of the combination of robust, highly accurate machine learning algorithms with linear scalability that renders the applications of these algorithms to massive astronomical data tractable. We demonstrate the core algorithms kernel density estimation, K-means clustering, linear regression, nearest neighbors, random forest and gradient-boosted decision tree, singular value decomposition, support vector machine, and two-point correlation function. Each of these is relevant for astronomical applications such as finding novel astrophysical objects, characterizing artifacts in data, object classification (including for rare objects), object distances, finding the important features describing objects, density estimation of distributions, probabilistic quantities, and exploring the unknown structure of new data. The software, Skytree Server, runs on any UNIX-based machine, a virtual machine, or cloud-based and distributed systems including Hadoop. We have integrated it on the cloud computing system of the Canadian Astronomical Data Centre, the Canadian Advanced Network for Astronomical Research (CANFAR), creating the world's first cloud computing data mining system for astronomy. We demonstrate results showing the scaling of each of our major algorithms on large astronomical datasets, including the full 470,992,970 objects of the 2 Micron All-Sky Survey (2MASS) Point Source Catalog. We demonstrate the ability to find outliers in the full 2MASS dataset utilizing multiple methods, e.g., nearest neighbors, and the local outlier factor. 2MASS is used as a proof-of-concept dataset due to its convenience and availability. These results are of interest to any astronomical project with large and/or complex datasets that wishes to extract the full scientific value from its data.

Algorithm for Detection of Ground and Canopy Cover in Micropulse Photon-Counting Lidar Altimeter Data in Preparation for the ICESat-2 Mission

NASA Technical Reports Server (NTRS)

Herzfeld, Ute Christina; McDonald, Brian W.; Neumann, Thomas Allen; Wallin, Bruce F.; Neumann, Thomas A.; Markus, Thorsten; Brenner, Anita; Field, Christopher

2014-01-01

NASA's Ice, Cloud and Land Elevation Satellite-II (ICESat-2) mission is a decadal survey mission (2016 launch). The mission objectives are to measure land ice elevation, sea ice freeboard, and changes in these variables, as well as to collect measurements over vegetation to facilitate canopy height determination. Two innovative components will characterize the ICESat-2 lidar: 1) collection of elevation data by a multibeam system and 2) application of micropulse lidar (photon-counting) technology. A photon-counting altimeter yields clouds of discrete points, resulting from returns of individual photons, and hence new data analysis techniques are required for elevation determination and association of the returned points to reflectors of interest. The objective of this paper is to derive an algorithm that allows detection of ground under dense canopy and identification of ground and canopy levels in simulated ICESat-2 data, based on airborne observations with a Sigma Space micropulse lidar. The mathematical algorithm uses spatial statistical and discrete mathematical concepts, including radial basis functions, density measures, geometrical anisotropy, eigenvectors, and geostatistical classification parameters and hyperparameters. Validation shows that ground and canopy elevation, and hence canopy height, can be expected to be observable with high accuracy by ICESat-2 for all expected beam energies considered for instrument design (93.01%-99.57% correctly selected points for a beam with expected return of 0.93 mean signals per shot (msp), and 72.85%-98.68% for 0.48 msp). The algorithm derived here is generally applicable for elevation determination from photoncounting lidar altimeter data collected over forested areas, land ice, sea ice, and land surfaces, as well as for cloud detection.

Cloud Radiative Effect in dependence on Cloud Type

NASA Astrophysics Data System (ADS)

Aebi, Christine; Gröbner, Julian; Kämpfer, Niklaus; Vuilleumier, Laurent

2015-04-01

Radiative transfer of energy in the atmosphere and the influence of clouds on the radiation budget remain the greatest sources of uncertainty in the simulation of climate change. Small changes in cloudiness and radiation can have large impacts on the Earth's climate. In order to assess the opposing effects of clouds on the radiation budget and the corresponding changes, frequent and more precise radiation and cloud observations are necessary. The role of clouds on the surface radiation budget is studied in order to quantify the longwave, shortwave and the total cloud radiative forcing in dependence on the atmospheric composition and cloud type. The study is performed for three different sites in Switzerland at three different altitude levels: Payerne (490 m asl), Davos (1'560 m asl) and Jungfraujoch (3'580 m asl). On the basis of data of visible all-sky camera systems at the three aforementioned stations in Switzerland, up to six different cloud types are distinguished (Cirrus-Cirrostratus, Cirrocumulus-Altocumulus, Stratus-Altostratus, Cumulus, Stratocumulus and Cumulonimbus-Nimbostratus). These cloud types are classified with a modified algorithm of Heinle et al. (2010). This cloud type classifying algorithm is based on a set of statistical features describing the color (spectral features) and the texture of an image (textural features) (Wacker et al. (2015)). The calculation of the fractional cloud cover information is based on spectral information of the all-sky camera data. The radiation data are taken from measurements with pyranometers and pyrgeometers at the different stations. A climatology of a whole year of the shortwave, longwave and total cloud radiative effect and its sensitivity to integrated water vapor, cloud cover and cloud type will be calculated for the three above-mentioned stations in Switzerland. For the calculation of the shortwave and longwave cloud radiative effect the corresponding cloud-free reference models developed at PMOD/WRC will be used (Wacker et al. (2013)). References: Heinle, A., A. Macke and A. Srivastav (2010) Automatic cloud classification of whole sky images, Atmospheric Measurement Techniques. Wacker, S., J. Gröbner and L. Vuilleumier (2013) A method to calculate cloud-free long-wave irradiance at the surface based on radiative transfer modeling and temperature lapse rate estimates, Theoretical and Applied Climatology. Wacker, S., J. Gröbner, C. Zysset, L. Diener, P. Tzoumanikis, A. Kazantzidis, L. Vuilleumier, R. Stöckli, S. Nyeki, and N. Kämpfer (2015) Cloud observations in Switzerland using hemispherical sky cameras, Journal of Geophysical Research.

Cloud Model Bat Algorithm

PubMed Central

Zhou, Yongquan; Xie, Jian; Li, Liangliang; Ma, Mingzhi

2014-01-01

Bat algorithm (BA) is a novel stochastic global optimization algorithm. Cloud model is an effective tool in transforming between qualitative concepts and their quantitative representation. Based on the bat echolocation mechanism and excellent characteristics of cloud model on uncertainty knowledge representation, a new cloud model bat algorithm (CBA) is proposed. This paper focuses on remodeling echolocation model based on living and preying characteristics of bats, utilizing the transformation theory of cloud model to depict the qualitative concept: “bats approach their prey.” Furthermore, Lévy flight mode and population information communication mechanism of bats are introduced to balance the advantage between exploration and exploitation. The simulation results show that the cloud model bat algorithm has good performance on functions optimization. PMID:24967425

An automated cirrus classification

NASA Astrophysics Data System (ADS)

Gryspeerdt, Edward; Quaas, Johannes; Goren, Tom; Klocke, Daniel; Brueck, Matthias

2018-05-01

Cirrus clouds play an important role in determining the radiation budget of the earth, but many of their properties remain uncertain, particularly their response to aerosol variations and to warming. Part of the reason for this uncertainty is the dependence of cirrus cloud properties on the cloud formation mechanism, which itself is strongly dependent on the local meteorological conditions. In this work, a classification system (Identification and Classification of Cirrus or IC-CIR) is introduced to identify cirrus clouds by the cloud formation mechanism. Using reanalysis and satellite data, cirrus clouds are separated into four main types: orographic, frontal, convective and synoptic. Through a comparison to convection-permitting model simulations and back-trajectory-based analysis, it is shown that these observation-based regimes can provide extra information on the cloud-scale updraughts and the frequency of occurrence of liquid-origin ice, with the convective regime having higher updraughts and a greater occurrence of liquid-origin ice compared to the synoptic regimes. Despite having different cloud formation mechanisms, the radiative properties of the regimes are not distinct, indicating that retrieved cloud properties alone are insufficient to completely describe them. This classification is designed to be easily implemented in GCMs, helping improve future model-observation comparisons and leading to improved parametrisations of cirrus cloud processes.

A building extraction approach for Airborne Laser Scanner data utilizing the Object Based Image Analysis paradigm

NASA Astrophysics Data System (ADS)

Tomljenovic, Ivan; Tiede, Dirk; Blaschke, Thomas

2016-10-01

In the past two decades Object-Based Image Analysis (OBIA) established itself as an efficient approach for the classification and extraction of information from remote sensing imagery and, increasingly, from non-image based sources such as Airborne Laser Scanner (ALS) point clouds. ALS data is represented in the form of a point cloud with recorded multiple returns and intensities. In our work, we combined OBIA with ALS point cloud data in order to identify and extract buildings as 2D polygons representing roof outlines in a top down mapping approach. We performed rasterization of the ALS data into a height raster for the purpose of the generation of a Digital Surface Model (DSM) and a derived Digital Elevation Model (DEM). Further objects were generated in conjunction with point statistics from the linked point cloud. With the use of class modelling methods, we generated the final target class of objects representing buildings. The approach was developed for a test area in Biberach an der Riß (Germany). In order to point out the possibilities of the adaptation-free transferability to another data set, the algorithm has been applied ;as is; to the ISPRS Benchmarking data set of Toronto (Canada). The obtained results show high accuracies for the initial study area (thematic accuracies of around 98%, geometric accuracy of above 80%). The very high performance within the ISPRS Benchmark without any modification of the algorithm and without any adaptation of parameters is particularly noteworthy.

Epilepsy analytic system with cloud computing.

PubMed

Shen, Chia-Ping; Zhou, Weizhi; Lin, Feng-Seng; Sung, Hsiao-Ya; Lam, Yan-Yu; Chen, Wei; Lin, Jeng-Wei; Pan, Ming-Kai; Chiu, Ming-Jang; Lai, Feipei

2013-01-01

Biomedical data analytic system has played an important role in doing the clinical diagnosis for several decades. Today, it is an emerging research area of analyzing these big data to make decision support for physicians. This paper presents a parallelized web-based tool with cloud computing service architecture to analyze the epilepsy. There are many modern analytic functions which are wavelet transform, genetic algorithm (GA), and support vector machine (SVM) cascaded in the system. To demonstrate the effectiveness of the system, it has been verified by two kinds of electroencephalography (EEG) data, which are short term EEG and long term EEG. The results reveal that our approach achieves the total classification accuracy higher than 90%. In addition, the entire training time accelerate about 4.66 times and prediction time is also meet requirements in real time.

Classification of Informal Settlements Through the Integration of 2d and 3d Features Extracted from Uav Data

NASA Astrophysics Data System (ADS)

Gevaert, C. M.; Persello, C.; Sliuzas, R.; Vosselman, G.

2016-06-01

Unmanned Aerial Vehicles (UAVs) are capable of providing very high resolution and up-to-date information to support informal settlement upgrading projects. In order to provide accurate basemaps, urban scene understanding through the identification and classification of buildings and terrain is imperative. However, common characteristics of informal settlements such as small, irregular buildings with heterogeneous roof material and large presence of clutter challenge state-of-the-art algorithms. Especially the dense buildings and steeply sloped terrain cause difficulties in identifying elevated objects. This work investigates how 2D radiometric and textural features, 2.5D topographic features, and 3D geometric features obtained from UAV imagery can be integrated to obtain a high classification accuracy in challenging classification problems for the analysis of informal settlements. It compares the utility of pixel-based and segment-based features obtained from an orthomosaic and DSM with point-based and segment-based features extracted from the point cloud to classify an unplanned settlement in Kigali, Rwanda. Findings show that the integration of 2D and 3D features leads to higher classification accuracies.

Cloud Type Classification (cldtype) Value-Added Product

DOE Office of Scientific and Technical Information (OSTI.GOV)

Flynn, Donna; Shi, Yan; Lim, K-S

The Cloud Type (cldtype) value-added product (VAP) provides an automated cloud type classification based on macrophysical quantities derived from vertically pointing lidar and radar. Up to 10 layers of clouds are classified into seven cloud types based on predetermined and site-specific thresholds of cloud top, base and thickness. Examples of thresholds for selected U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Climate Research Facility sites are provided in Tables 1 and 2. Inputs for the cldtype VAP include lidar and radar cloud boundaries obtained from the Active Remotely Sensed Cloud Location (ARSCL) and Surface Meteorological Systems (MET) data. Rainmore » rates from MET are used to determine when radar signal attenuation precludes accurate cloud detection. Temporal resolution and vertical resolution for cldtype are 1 minute and 30 m respectively and match the resolution of ARSCL. The cldtype classification is an initial step for further categorization of clouds. It was developed for use by the Shallow Cumulus VAP to identify potential periods of interest to the LASSO model and is intended to find clouds of interest for a variety of users.« less

Pattern recognition of satellite cloud imagery for improved weather prediction

NASA Technical Reports Server (NTRS)

Gautier, Catherine; Somerville, Richard C. J.; Volfson, Leonid B.

1986-01-01

The major accomplishment was the successful development of a method for extracting time derivative information from geostationary meteorological satellite imagery. This research is a proof-of-concept study which demonstrates the feasibility of using pattern recognition techniques and a statistical cloud classification method to estimate time rate of change of large-scale meteorological fields from remote sensing data. The cloud classification methodology is based on typical shape function analysis of parameter sets characterizing the cloud fields. The three specific technical objectives, all of which were successfully achieved, are as follows: develop and test a cloud classification technique based on pattern recognition methods, suitable for the analysis of visible and infrared geostationary satellite VISSR imagery; develop and test a methodology for intercomparing successive images using the cloud classification technique, so as to obtain estimates of the time rate of change of meteorological fields; and implement this technique in a testbed system incorporating an interactive graphics terminal to determine the feasibility of extracting time derivative information suitable for comparison with numerical weather prediction products.

Cloud cover detection combining high dynamic range sky images and ceilometer measurements

NASA Astrophysics Data System (ADS)

Román, R.; Cazorla, A.; Toledano, C.; Olmo, F. J.; Cachorro, V. E.; de Frutos, A.; Alados-Arboledas, L.

2017-11-01

This paper presents a new algorithm for cloud detection based on high dynamic range images from a sky camera and ceilometer measurements. The algorithm is also able to detect the obstruction of the sun. This algorithm, called CPC (Camera Plus Ceilometer), is based on the assumption that under cloud-free conditions the sky field must show symmetry. The symmetry criteria are applied depending on ceilometer measurements of the cloud base height. CPC algorithm is applied in two Spanish locations (Granada and Valladolid). The performance of CPC retrieving the sun conditions (obstructed or unobstructed) is analyzed in detail using as reference pyranometer measurements at Granada. CPC retrievals are in agreement with those derived from the reference pyranometer in 85% of the cases (it seems that this agreement does not depend on aerosol size or optical depth). The agreement percentage goes down to only 48% when another algorithm, based on Red-Blue Ratio (RBR), is applied to the sky camera images. The retrieved cloud cover at Granada and Valladolid is compared with that registered by trained meteorological observers. CPC cloud cover is in agreement with the reference showing a slight overestimation and a mean absolute error around 1 okta. A major advantage of the CPC algorithm with respect to the RBR method is that the determined cloud cover is independent of aerosol properties. The RBR algorithm overestimates cloud cover for coarse aerosols and high loads. Cloud cover obtained only from ceilometer shows similar results than CPC algorithm; but the horizontal distribution cannot be obtained. In addition, it has been observed that under quick and strong changes on cloud cover ceilometers retrieve a cloud cover fitting worse with the real cloud cover.

Improving Scene Classifications with Combined Active/Passive Measurements

NASA Astrophysics Data System (ADS)

Hu, Y.; Rodier, S.; Vaughan, M.; McGill, M.

The uncertainties in cloud and aerosol physical properties derived from passive instruments such as MODIS are not insignificant And the uncertainty increases when the optical depths decrease Lidar observations do much better for the thin clouds and aerosols Unfortunately space-based lidar measurements such as the one onboard CALIPSO satellites are limited to nadir view only and thus have limited spatial coverage To produce climatologically meaningful thin cloud and aerosol data products it is necessary to combine the spatial coverage of MODIS with the highly sensitive CALIPSO lidar measurements Can we improving the quality of cloud and aerosol remote sensing data products by extending the knowledge about thin clouds and aerosols learned from CALIPSO-type of lidar measurements to a larger portion of the off-nadir MODIS-like multi-spectral pixels To answer the question we studied the collocated Cloud Physics Lidar CPL with Modis-Airborne-Simulation MAS observations and established an effective data fusion technique that will be applied in the combined CALIPSO MODIS cloud aerosol product algorithms This technique performs k-mean and Kohonen self-organized map cluster analysis on the entire swath of MAS data as well as on the combined CPL MAS data at the nadir track Interestingly the clusters generated from the two approaches are almost identical It indicates that the MAS multi-spectral data may have already captured most of the cloud and aerosol scene types such as cloud ice water phase multi-layer information aerosols

Cloud Impacts on Pavement Temperature in Energy Balance Models

NASA Astrophysics Data System (ADS)

Walker, C. L.

2013-12-01

Forecast systems provide decision support for end-users ranging from the solar energy industry to municipalities concerned with road safety. Pavement temperature is an important variable when considering vehicle response to various weather conditions. A complex, yet direct relationship exists between tire and pavement temperatures. Literature has shown that as tire temperature increases, friction decreases which affects vehicle performance. Many forecast systems suffer from inaccurate radiation forecasts resulting in part from the inability to model different types of clouds and their influence on radiation. This research focused on forecast improvement by determining how cloud type impacts the amount of shortwave radiation reaching the surface and subsequent pavement temperatures. The study region was the Great Plains where surface solar radiation data were obtained from the High Plains Regional Climate Center's Automated Weather Data Network stations. Road pavement temperature data were obtained from the Meteorological Assimilation Data Ingest System. Cloud properties and radiative transfer quantities were obtained from the Clouds and Earth's Radiant Energy System mission via Aqua and Terra Moderate Resolution Imaging Spectroradiometer satellite products. An additional cloud data set was incorporated from the Naval Research Laboratory Cloud Classification algorithm. Statistical analyses using a modified nearest neighbor approach were first performed relating shortwave radiation variability with road pavement temperature fluctuations. Then statistical associations were determined between the shortwave radiation and cloud property data sets. Preliminary results suggest that substantial pavement forecasting improvement is possible with the inclusion of cloud-specific information. Future model sensitivity testing seeks to quantify the magnitude of forecast improvement.

Ten Years of Cloud Optical and Microphysical Retrievals from MODIS

NASA Technical Reports Server (NTRS)

Platnick, Steven; King, Michael D.; Wind, Galina; Hubanks, Paul; Arnold, G. Thomas; Amarasinghe, Nandana

2010-01-01

The MODIS cloud optical properties algorithm (MOD06/MYD06 for Terra and Aqua MODIS, respectively) has undergone extensive improvements and enhancements since the launch of Terra. These changes have included: improvements in the cloud thermodynamic phase algorithm; substantial changes in the ice cloud light scattering look up tables (LUTs); a clear-sky restoral algorithm for flagging heavy aerosol and sunglint; greatly improved spectral surface albedo maps, including the spectral albedo of snow by ecosystem; inclusion of pixel-level uncertainty estimates for cloud optical thickness, effective radius, and water path derived for three error sources that includes the sensitivity of the retrievals to solar and viewing geometries. To improve overall retrieval quality, we have also implemented cloud edge removal and partly cloudy detection (using MOD35 cloud mask 250m tests), added a supplementary cloud optical thickness and effective radius algorithm over snow and sea ice surfaces and over the ocean, which enables comparison with the "standard" 2.1 11m effective radius retrieval, and added a multi-layer cloud detection algorithm. We will discuss the status of the MOD06 algorithm and show examples of pixellevel (Level-2) cloud retrievals for selected data granules, as well as gridded (Level-3) statistics, notably monthly means and histograms (lD and 2D, with the latter giving correlations between cloud optical thickness and effective radius, and other cloud product pairs).

Cloud retrievals from satellite data using optimal estimation: evaluation and application to ATSR

NASA Astrophysics Data System (ADS)

Poulsen, C. A.; Siddans, R.; Thomas, G. E.; Sayer, A. M.; Grainger, R. G.; Campmany, E.; Dean, S. M.; Arnold, C.; Watts, P. D.

2012-08-01

Clouds play an important role in balancing the Earth's radiation budget. Hence, it is vital that cloud climatologies are produced that quantify cloud macro and micro physical parameters and the associated uncertainty. In this paper, we present an algorithm ORAC (Oxford-RAL retrieval of Aerosol and Cloud) which is based on fitting a physically consistent cloud model to satellite observations simultaneously from the visible to the mid-infrared, thereby ensuring that the resulting cloud properties provide both a good representation of the short-wave and long-wave radiative effects of the observed cloud. The advantages of the optimal estimation method are that it enables rigorous error propagation and the inclusion of all measurements and any a priori information and associated errors in a rigorous mathematical framework. The algorithm provides a measure of the consistency between retrieval representation of cloud and satellite radiances. The cloud parameters retrieved are the cloud top pressure, cloud optical depth, cloud effective radius, cloud fraction and cloud phase. The algorithm can be applied to most visible/infrared satellite instruments. In this paper, we demonstrate the applicability to the Along-Track Scanning Radiometers ATSR-2 and AATSR. Examples of applying the algorithm to ATSR-2 flight data are presented and the sensitivity of the retrievals assessed, in particular the algorithm is evaluated for a number of simulated single-layer and multi-layer conditions. The algorithm was found to perform well for single-layer cloud except when the cloud was very thin; i.e., less than 1 optical depths. For the multi-layer cloud, the algorithm was robust except when the upper ice cloud layer is less than five optical depths. In these cases the retrieved cloud top pressure and cloud effective radius become a weighted average of the 2 layers. The sum of optical depth of multi-layer cloud is retrieved well until the cloud becomes thick, greater than 50 optical depths, where the cloud begins to saturate. The cost proved a good indicator of multi-layer scenarios. Both the retrieval cost and the error need to be considered together in order to evaluate the quality of the retrieval. This algorithm in the configuration described here has been applied to both ATSR-2 and AATSR visible and infrared measurements in the context of the GRAPE (Global Retrieval and cloud Product Evaluation) project to produce a 14 yr consistent record for climate research.

Joint classification and contour extraction of large 3D point clouds

NASA Astrophysics Data System (ADS)

Hackel, Timo; Wegner, Jan D.; Schindler, Konrad

2017-08-01

We present an effective and efficient method for point-wise semantic classification and extraction of object contours of large-scale 3D point clouds. What makes point cloud interpretation challenging is the sheer size of several millions of points per scan and the non-grid, sparse, and uneven distribution of points. Standard image processing tools like texture filters, for example, cannot handle such data efficiently, which calls for dedicated point cloud labeling methods. It turns out that one of the major drivers for efficient computation and handling of strong variations in point density, is a careful formulation of per-point neighborhoods at multiple scales. This allows, both, to define an expressive feature set and to extract topologically meaningful object contours. Semantic classification and contour extraction are interlaced problems. Point-wise semantic classification enables extracting a meaningful candidate set of contour points while contours help generating a rich feature representation that benefits point-wise classification. These methods are tailored to have fast run time and small memory footprint for processing large-scale, unstructured, and inhomogeneous point clouds, while still achieving high classification accuracy. We evaluate our methods on the semantic3d.net benchmark for terrestrial laser scans with >109 points.

Cloud computing task scheduling strategy based on improved differential evolution algorithm

NASA Astrophysics Data System (ADS)

Ge, Junwei; He, Qian; Fang, Yiqiu

2017-04-01

In order to optimize the cloud computing task scheduling scheme, an improved differential evolution algorithm for cloud computing task scheduling is proposed. Firstly, the cloud computing task scheduling model, according to the model of the fitness function, and then used improved optimization calculation of the fitness function of the evolutionary algorithm, according to the evolution of generation of dynamic selection strategy through dynamic mutation strategy to ensure the global and local search ability. The performance test experiment was carried out in the CloudSim simulation platform, the experimental results show that the improved differential evolution algorithm can reduce the cloud computing task execution time and user cost saving, good implementation of the optimal scheduling of cloud computing tasks.

Automated extraction and analysis of rock discontinuity characteristics from 3D point clouds

NASA Astrophysics Data System (ADS)

Bianchetti, Matteo; Villa, Alberto; Agliardi, Federico; Crosta, Giovanni B.

2016-04-01

A reliable characterization of fractured rock masses requires an exhaustive geometrical description of discontinuities, including orientation, spacing, and size. These are required to describe discontinuum rock mass structure, perform Discrete Fracture Network and DEM modelling, or provide input for rock mass classification or equivalent continuum estimate of rock mass properties. Although several advanced methodologies have been developed in the last decades, a complete characterization of discontinuity geometry in practice is still challenging, due to scale-dependent variability of fracture patterns and difficult accessibility to large outcrops. Recent advances in remote survey techniques, such as terrestrial laser scanning and digital photogrammetry, allow a fast and accurate acquisition of dense 3D point clouds, which promoted the development of several semi-automatic approaches to extract discontinuity features. Nevertheless, these often need user supervision on algorithm parameters which can be difficult to assess. To overcome this problem, we developed an original Matlab tool, allowing fast, fully automatic extraction and analysis of discontinuity features with no requirements on point cloud accuracy, density and homogeneity. The tool consists of a set of algorithms which: (i) process raw 3D point clouds, (ii) automatically characterize discontinuity sets, (iii) identify individual discontinuity surfaces, and (iv) analyse their spacing and persistence. The tool operates in either a supervised or unsupervised mode, starting from an automatic preliminary exploration data analysis. The identification and geometrical characterization of discontinuity features is divided in steps. First, coplanar surfaces are identified in the whole point cloud using K-Nearest Neighbor and Principal Component Analysis algorithms optimized on point cloud accuracy and specified typical facet size. Then, discontinuity set orientation is calculated using Kernel Density Estimation and principal vector similarity criteria. Poles to points are assigned to individual discontinuity objects using easy custom vector clustering and Jaccard distance approaches, and each object is segmented into planar clusters using an improved version of the DBSCAN algorithm. Modal set orientations are then recomputed by cluster-based orientation statistics to avoid the effects of biases related to cluster size and density heterogeneity of the point cloud. Finally, spacing values are measured between individual discontinuity clusters along scanlines parallel to modal pole vectors, whereas individual feature size (persistence) is measured using 3D convex hull bounding boxes. Spacing and size are provided both as raw population data and as summary statistics. The tool is optimized for parallel computing on 64bit systems, and a Graphic User Interface (GUI) has been developed to manage data processing, provide several outputs, including reclassified point clouds, tables, plots, derived fracture intensity parameters, and export to modelling software tools. We present test applications performed both on synthetic 3D data (simple 3D solids) and real case studies, validating the results with existing geomechanical datasets.

Improving the Accuracy of Cloud Detection Using Machine Learning

NASA Astrophysics Data System (ADS)

Craddock, M. E.; Alliss, R. J.; Mason, M.

2017-12-01

Cloud detection from geostationary satellite imagery has long been accomplished through multi-spectral channel differencing in comparison to the Earth's surface. The distinction of clear/cloud is then determined by comparing these differences to empirical thresholds. Using this methodology, the probability of detecting clouds exceeds 90% but performance varies seasonally, regionally and temporally. The Cloud Mask Generator (CMG) database developed under this effort, consists of 20 years of 4 km, 15minute clear/cloud images based on GOES data over CONUS and Hawaii. The algorithms to determine cloudy pixels in the imagery are based on well-known multi-spectral techniques and defined thresholds. These thresholds were produced by manually studying thousands of images and thousands of man-hours to determine the success and failure of the algorithms to fine tune the thresholds. This study aims to investigate the potential of improving cloud detection by using Random Forest (RF) ensemble classification. RF is the ideal methodology to employ for cloud detection as it runs efficiently on large datasets, is robust to outliers and noise and is able to deal with highly correlated predictors, such as multi-spectral satellite imagery. The RF code was developed using Python in about 4 weeks. The region of focus selected was Hawaii and includes the use of visible and infrared imagery, topography and multi-spectral image products as predictors. The development of the cloud detection technique is realized in three steps. First, tuning of the RF models is completed to identify the optimal values of the number of trees and number of predictors to employ for both day and night scenes. Second, the RF models are trained using the optimal number of trees and a select number of random predictors identified during the tuning phase. Lastly, the model is used to predict clouds for an independent time period than used during training and compared to truth, the CMG cloud mask. Initial results show 97% accuracy during the daytime, 94% accuracy at night, and 95% accuracy for all times. The total time to train, tune and test was approximately one week. The improved performance and reduced time to produce results is testament to improved computer technology and the use of machine learning as a more efficient and accurate methodology of cloud detection.

A cloud masking algorithm for EARLINET lidar systems

NASA Astrophysics Data System (ADS)

Binietoglou, Ioannis; Baars, Holger; D'Amico, Giuseppe; Nicolae, Doina

2015-04-01

Cloud masking is an important first step in any aerosol lidar processing chain as most data processing algorithms can only be applied on cloud free observations. Up to now, the selection of a cloud-free time interval for data processing is typically performed manually, and this is one of the outstanding problems for automatic processing of lidar data in networks such as EARLINET. In this contribution we present initial developments of a cloud masking algorithm that permits the selection of the appropriate time intervals for lidar data processing based on uncalibrated lidar signals. The algorithm is based on a signal normalization procedure using the range of observed values of lidar returns, designed to work with different lidar systems with minimal user input. This normalization procedure can be applied to measurement periods of only few hours, even if no suitable cloud-free interval exists, and thus can be used even when only a short period of lidar measurements is available. Clouds are detected based on a combination of criteria including the magnitude of the normalized lidar signal and time-space edge detection performed using the Sobel operator. In this way the algorithm avoids misclassification of strong aerosol layers as clouds. Cloud detection is performed using the highest available time and vertical resolution of the lidar signals, allowing the effective detection of low-level clouds (e.g. cumulus humilis). Special attention is given to suppress false cloud detection due to signal noise that can affect the algorithm's performance, especially during day-time. In this contribution we present the details of algorithm, the effect of lidar characteristics (space-time resolution, available wavelengths, signal-to-noise ratio) to detection performance, and highlight the current strengths and limitations of the algorithm using lidar scenes from different lidar systems in different locations across Europe.

The operational cloud retrieval algorithms from TROPOMI on board Sentinel-5 Precursor

NASA Astrophysics Data System (ADS)

Loyola, Diego G.; Gimeno García, Sebastián; Lutz, Ronny; Argyrouli, Athina; Romahn, Fabian; Spurr, Robert J. D.; Pedergnana, Mattia; Doicu, Adrian; Molina García, Víctor; Schüssler, Olena

2018-01-01

This paper presents the operational cloud retrieval algorithms for the TROPOspheric Monitoring Instrument (TROPOMI) on board the European Space Agency Sentinel-5 Precursor (S5P) mission scheduled for launch in 2017. Two algorithms working in tandem are used for retrieving cloud properties: OCRA (Optical Cloud Recognition Algorithm) and ROCINN (Retrieval of Cloud Information using Neural Networks). OCRA retrieves the cloud fraction using TROPOMI measurements in the ultraviolet (UV) and visible (VIS) spectral regions, and ROCINN retrieves the cloud top height (pressure) and optical thickness (albedo) using TROPOMI measurements in and around the oxygen A-band in the near infrared (NIR). Cloud parameters from TROPOMI/S5P will be used not only for enhancing the accuracy of trace gas retrievals but also for extending the satellite data record of cloud information derived from oxygen A-band measurements, a record initiated with the Global Ozone Monitoring Experiment (GOME) on board the second European Remote-Sensing Satellite (ERS-2) over 20 years ago. The OCRA and ROCINN algorithms are integrated in the S5P operational processor UPAS (Universal Processor for UV/VIS/NIR Atmospheric Spectrometers), and we present here UPAS cloud results using the Ozone Monitoring Instrument (OMI) and GOME-2 measurements. In addition, we examine anticipated challenges for the TROPOMI/S5P cloud retrieval algorithms, and we discuss the future validation needs for OCRA and ROCINN.

Cloud-scale genomic signals processing classification analysis for gene expression microarray data.

PubMed

Harvey, Benjamin; Soo-Yeon Ji

2014-01-01

As microarray data available to scientists continues to increase in size and complexity, it has become overwhelmingly important to find multiple ways to bring inference though analysis of DNA/mRNA sequence data that is useful to scientists. Though there have been many attempts to elucidate the issue of bringing forth biological inference by means of wavelet preprocessing and classification, there has not been a research effort that focuses on a cloud-scale classification analysis of microarray data using Wavelet thresholding in a Cloud environment to identify significantly expressed features. This paper proposes a novel methodology that uses Wavelet based Denoising to initialize a threshold for determination of significantly expressed genes for classification. Additionally, this research was implemented and encompassed within cloud-based distributed processing environment. The utilization of Cloud computing and Wavelet thresholding was used for the classification 14 tumor classes from the Global Cancer Map (GCM). The results proved to be more accurate than using a predefined p-value for differential expression classification. This novel methodology analyzed Wavelet based threshold features of gene expression in a Cloud environment, furthermore classifying the expression of samples by analyzing gene patterns, which inform us of biological processes. Moreover, enabling researchers to face the present and forthcoming challenges that may arise in the analysis of data in functional genomics of large microarray datasets.

Applicability Analysis of Cloth Simulation Filtering Algorithm for Mobile LIDAR Point Cloud

NASA Astrophysics Data System (ADS)

Cai, S.; Zhang, W.; Qi, J.; Wan, P.; Shao, J.; Shen, A.

2018-04-01

Classifying the original point clouds into ground and non-ground points is a key step in LiDAR (light detection and ranging) data post-processing. Cloth simulation filtering (CSF) algorithm, which based on a physical process, has been validated to be an accurate, automatic and easy-to-use algorithm for airborne LiDAR point cloud. As a new technique of three-dimensional data collection, the mobile laser scanning (MLS) has been gradually applied in various fields, such as reconstruction of digital terrain models (DTM), 3D building modeling and forest inventory and management. Compared with airborne LiDAR point cloud, there are some different features (such as point density feature, distribution feature and complexity feature) for mobile LiDAR point cloud. Some filtering algorithms for airborne LiDAR data were directly used in mobile LiDAR point cloud, but it did not give satisfactory results. In this paper, we explore the ability of the CSF algorithm for mobile LiDAR point cloud. Three samples with different shape of the terrain are selected to test the performance of this algorithm, which respectively yields total errors of 0.44 %, 0.77 % and1.20 %. Additionally, large area dataset is also tested to further validate the effectiveness of this algorithm, and results show that it can quickly and accurately separate point clouds into ground and non-ground points. In summary, this algorithm is efficient and reliable for mobile LiDAR point cloud.

Cloud cover analysis with Arctic Advanced Very High Resolution Radiometer data. II - Classification with spectral and textural measures

NASA Technical Reports Server (NTRS)

Key, J.

1990-01-01

The spectral and textural characteristics of polar clouds and surfaces for a 7-day summer series of AVHRR data in two Arctic locations are examined, and the results used in the development of a cloud classification procedure for polar satellite data. Since spatial coherence and texture sensitivity tests indicate that a joint spectral-textural analysis based on the same cell size is inappropriate, cloud detection with AVHRR data and surface identification with passive microwave data are first done on the pixel level as described by Key and Barry (1989). Next, cloud patterns within 250-sq-km regions are described, then the spectral and local textural characteristics of cloud patterns in the image are determined and each cloud pixel is classified by statistical methods. Results indicate that both spectral and textural features can be utilized in the classification of cloudy pixels, although spectral features are most useful for the discrimination between cloud classes.

Mapping forest tree species over large areas with partially cloudy Landsat imagery

NASA Astrophysics Data System (ADS)

Turlej, K.; Radeloff, V.

2017-12-01

Forests provide numerous services to natural systems and humankind, but which services forest provide depends greatly on their tree species composition. That makes it important to track not only changes in forest extent, something that remote sensing excels in, but also to map tree species. The main goal of our work was to map tree species with Landsat imagery, and to identify how to maximize mapping accuracy by including partially cloudy imagery. Our study area covered one Landsat footprint (26/28) in Northern Wisconsin, USA, with temperate and boreal forests. We selected this area because it contains numerous tree species and variable forest composition providing an ideal study area to test the limits of Landsat data. We quantified how species-level classification accuracy was affected by a) the number of acquisitions, b) the seasonal distribution of observations, and c) the amount of cloud contamination. We classified a single year stack of Landsat-7, and -8 images data with a decision tree algorithm to generate a map of dominant tree species at the pixel- and stand-level. We obtained three important results. First, we achieved producer's accuracies in the range 70-80% and user's accuracies in range 80-90% for the most abundant tree species in our study area. Second, classification accuracy improved with more acquisitions, when observations were available from all seasons, and is the best when images with up to 40% cloud cover are included. Finally, classifications for pure stands were 10 to 30 percentage points better than those for mixed stands. We conclude that including partially cloudy Landsat imagery allows to map forest tree species with accuracies that were previously only possible for rare years with many cloud-free observations. Our approach thus provides important information for both forest management and science.

Implementation on Landsat Data of a Simple Cloud Mask Algorithm Developed for MODIS Land Bands

NASA Technical Reports Server (NTRS)

Oreopoulos, Lazaros; Wilson, Michael J.; Varnai, Tamas

2010-01-01

This letter assesses the performance on Landsat-7 images of a modified version of a cloud masking algorithm originally developed for clear-sky compositing of Moderate Resolution Imaging Spectroradiometer (MODIS) images at northern mid-latitudes. While data from recent Landsat missions include measurements at thermal wavelengths, and such measurements are also planned for the next mission, thermal tests are not included in the suggested algorithm in its present form to maintain greater versatility and ease of use. To evaluate the masking algorithm we take advantage of the availability of manual (visual) cloud masks developed at USGS for the collection of Landsat scenes used here. As part of our evaluation we also include the Automated Cloud Cover Assesment (ACCA) algorithm that includes thermal tests and is used operationally by the Landsat-7 mission to provide scene cloud fractions, but no cloud masks. We show that the suggested algorithm can perform about as well as ACCA both in terms of scene cloud fraction and pixel-level cloud identification. Specifically, we find that the algorithm gives an error of 1.3% for the scene cloud fraction of 156 scenes, and a root mean square error of 7.2%, while it agrees with the manual mask for 93% of the pixels, figures very similar to those from ACCA (1.2%, 7.1%, 93.7%).

Statistics-based optimization of the polarimetric radar hydrometeor classification algorithm and its application for a squall line in South China

NASA Astrophysics Data System (ADS)

Wu, Chong; Liu, Liping; Wei, Ming; Xi, Baozhu; Yu, Minghui

2018-03-01

A modified hydrometeor classification algorithm (HCA) is developed in this study for Chinese polarimetric radars. This algorithm is based on the U.S. operational HCA. Meanwhile, the methodology of statistics-based optimization is proposed including calibration checking, datasets selection, membership functions modification, computation thresholds modification, and effect verification. Zhuhai radar, the first operational polarimetric radar in South China, applies these procedures. The systematic bias of calibration is corrected, the reliability of radar measurements deteriorates when the signal-to-noise ratio is low, and correlation coefficient within the melting layer is usually lower than that of the U.S. WSR-88D radar. Through modification based on statistical analysis of polarimetric variables, the localized HCA especially for Zhuhai is obtained, and it performs well over a one-month test through comparison with sounding and surface observations. The algorithm is then utilized for analysis of a squall line process on 11 May 2014 and is found to provide reasonable details with respect to horizontal and vertical structures, and the HCA results—especially in the mixed rain-hail region—can reflect the life cycle of the squall line. In addition, the kinematic and microphysical processes of cloud evolution and the differences between radar-detected hail and surface observations are also analyzed. The results of this study provide evidence for the improvement of this HCA developed specifically for China.

ASTER cloud coverage reassessment using MODIS cloud mask products

NASA Astrophysics Data System (ADS)

Tonooka, Hideyuki; Omagari, Kunjuro; Yamamoto, Hirokazu; Tachikawa, Tetsushi; Fujita, Masaru; Paitaer, Zaoreguli

2010-10-01

In the Advanced Spaceborne Thermal Emission and Reflection radiometer (ASTER) Project, two kinds of algorithms are used for cloud assessment in Level-1 processing. The first algorithm based on the LANDSAT-5 TM Automatic Cloud Cover Assessment (ACCA) algorithm is used for a part of daytime scenes observed with only VNIR bands and all nighttime scenes, and the second algorithm based on the LANDSAT-7 ETM+ ACCA algorithm is used for most of daytime scenes observed with all spectral bands. However, the first algorithm does not work well for lack of some spectral bands sensitive to cloud detection, and the two algorithms have been less accurate over snow/ice covered areas since April 2008 when the SWIR subsystem developed troubles. In addition, they perform less well for some combinations of surface type and sun elevation angle. We, therefore, have developed the ASTER cloud coverage reassessment system using MODIS cloud mask (MOD35) products, and have reassessed cloud coverage for all ASTER archived scenes (>1.7 million scenes). All of the new cloud coverage data are included in Image Management System (IMS) databases of the ASTER Ground Data System (GDS) and NASA's Land Process Data Active Archive Center (LP DAAC) and used for ASTER product search by users, and cloud mask images are distributed to users through Internet. Daily upcoming scenes (about 400 scenes per day) are reassessed and inserted into the IMS databases in 5 to 7 days after each scene observation date. Some validation studies for the new cloud coverage data and some mission-related analyses using those data are also demonstrated in the present paper.

Validation of VIIRS Cloud Base Heights at Night Using Ground and Satellite Measurements over Alaska

NASA Astrophysics Data System (ADS)

NOH, Y. J.; Miller, S. D.; Seaman, C.; Forsythe, J. M.; Brummer, R.; Lindsey, D. T.; Walther, A.; Heidinger, A. K.; Li, Y.

2016-12-01

Knowledge of Cloud Base Height (CBH) is critical to describing cloud radiative feedbacks in numerical models and is of practical significance to aviation communities. We have developed a new CBH algorithm constrained by Cloud Top Height (CTH) and Cloud Water Path (CWP) by performing a statistical analysis of A-Train satellite data. It includes an extinction-based method for thin cirrus. In the algorithm, cloud geometric thickness is derived with upstream CTH and CWP input and subtracted from CTH to generate the topmost layer CBH. The CBH information is a key parameter for an improved Cloud Cover/Layers product. The algorithm has been applied to the Visible Infrared Imaging Radiometer Suite (VIIRS) onboard the Suomi NPP spacecraft. Nighttime cloud optical properties for CWP are retrieved from the nighttime lunar cloud optical and microphysical properties (NLCOMP) algorithm based on a lunar reflectance model for the VIIRS Day/Night Band (DNB) measuring nighttime visible light such as moonlight. The DNB has innovative capabilities to fill the polar winter and nighttime gap of cloud observations which has been an important shortfall from conventional radiometers. The CBH products have been intensively evaluated against CloudSat data. The results showed the new algorithm yields significantly improved performance over the original VIIRS CBH algorithm. However, since CloudSat is now operational during daytime only due to a battery anomaly, the nighttime performance has not been fully assessed. This presentation will show our approach to assess the performance of the CBH algorithm at night. VIIRS CBHs are retrieved over the Alaska region from October 2015 to April 2016 using the Clouds from AVHRR Extended (CLAVR-x) processing system. Ground-based measurements from ceilometer and micropulse lidar at the Atmospheric Radiation Measurement (ARM) site on the North Slope of Alaska are used for the analysis. Local weather conditions are checked using temperature and precipitation observations at the site. CALIPSO data with near-simultaneous colocation are added for multi-layered cloud cases which may have high clouds aloft beyond the ground measurements. Multi-month statistics of performance and case studies will be shown. Additional efforts for algorithm refinements will be also discussed.

Phenotype Instance Verification and Evaluation Tool (PIVET): A Scaled Phenotype Evidence Generation Framework Using Web-Based Medical Literature

PubMed Central

Ke, Junyuan; Ho, Joyce C; Ghosh, Joydeep; Wallace, Byron C

2018-01-01

Background Researchers are developing methods to automatically extract clinically relevant and useful patient characteristics from raw healthcare datasets. These characteristics, often capturing essential properties of patients with common medical conditions, are called computational phenotypes. Being generated by automated or semiautomated, data-driven methods, such potential phenotypes need to be validated as clinically meaningful (or not) before they are acceptable for use in decision making. Objective The objective of this study was to present Phenotype Instance Verification and Evaluation Tool (PIVET), a framework that uses co-occurrence analysis on an online corpus of publically available medical journal articles to build clinical relevance evidence sets for user-supplied phenotypes. PIVET adopts a conceptual framework similar to the pioneering prototype tool PheKnow-Cloud that was developed for the phenotype validation task. PIVET completely refactors each part of the PheKnow-Cloud pipeline to deliver vast improvements in speed without sacrificing the quality of the insights PheKnow-Cloud achieved. Methods PIVET leverages indexing in NoSQL databases to efficiently generate evidence sets. Specifically, PIVET uses a succinct representation of the phenotypes that corresponds to the index on the corpus database and an optimized co-occurrence algorithm inspired by the Aho-Corasick algorithm. We compare PIVET’s phenotype representation with PheKnow-Cloud’s by using PheKnow-Cloud’s experimental setup. In PIVET’s framework, we also introduce a statistical model trained on domain expert–verified phenotypes to automatically classify phenotypes as clinically relevant or not. Additionally, we show how the classification model can be used to examine user-supplied phenotypes in an online, rather than batch, manner. Results PIVET maintains the discriminative power of PheKnow-Cloud in terms of identifying clinically relevant phenotypes for the same corpus with which PheKnow-Cloud was originally developed, but PIVET’s analysis is an order of magnitude faster than that of PheKnow-Cloud. Not only is PIVET much faster, it can be scaled to a larger corpus and still retain speed. We evaluated multiple classification models on top of the PIVET framework and found ridge regression to perform best, realizing an average F1 score of 0.91 when predicting clinically relevant phenotypes. Conclusions Our study shows that PIVET improves on the most notable existing computational tool for phenotype validation in terms of speed and automation and is comparable in terms of accuracy. PMID:29728351

Improvement in thin cirrus retrievals using an emissivity-adjusted CO2 slicing algorithm

NASA Astrophysics Data System (ADS)

Zhang, Hong; Menzel, W. Paul

2002-09-01

CO2 slicing has been generally accepted as a useful algorithm for determining cloud top pressure (CTP) and effective cloud amount (ECA) for tropospheric clouds above 600 hPa. To date, the technique has assumed that the surface emissivity is that of a blackbody in the long-wavelength infrared radiances and that the cloud emissivities in spectrally close bands are approximately equal. The modified CO2 slicing algorithm considers adjustments of both surface emissivity and cloud emissivity ratio. Surface emissivity is adjusted according to the surface types. The ratio of cloud emissivities in spectrally close bands is adjusted away from unity according to radiative transfer calculations. The new CO2 slicing algorithm is examined with Moderate Resolution Imaging Spectroradiometer (MODIS) Airborne Simulator (MAS) CO2 band radiance measurements over thin clouds and validated against Cloud Lidar System (CLS) measurements of the same clouds; it is also applied to Geostationary Operational Environmental Satellite (GOES) Sounder data to study the overall impact on cloud property determinations. For high thin clouds an improved product emerges, while for thick and opaque clouds there is little change. For very thin clouds, the CTP increases by about 10-20 hPa and RMS (root mean square bias) difference is approximately 50 hPa; for thin clouds, the CTP increase is about 10 hPa bias and RMS difference is approximately 30 hPa. The new CO2 slicing algorithm places the clouds lower in the troposphere.

Evaluation of Multilayer Cloud Detection Using a MODIS CO2-Slicing Algorithm With CALIPSO-CloudSat Measurements

NASA Technical Reports Server (NTRS)

Viudez-Mora, Antonio; Kato, Seiji

2015-01-01

This work evaluates the multilayer cloud (MCF) algorithm based on CO2-slicing techniques against CALISPO-CloudSat (CLCS) measurement. This evaluation showed that the MCF underestimates the presence of multilayered clouds compared with CLCS and are retrained to cloud emissivities below 0.8 and cloud optical septs no larger than 0.3.

In situ measurements of angular-dependent light scattering by aerosols over the contiguous United States

NASA Astrophysics Data System (ADS)

Reed Espinosa, W.; Vanderlei Martins, J.; Remer, Lorraine A.; Puthukkudy, Anin; Orozco, Daniel; Dolgos, Gergely

2018-03-01

This work provides a synopsis of aerosol phase function (F11) and polarized phase function (F12) measurements made by the Polarized Imaging Nephelometer (PI-Neph) during the Studies of Emissions, Atmospheric Composition, Clouds and Climate Coupling by Regional Surveys (SEAC4RS) and the Deep Convection Clouds and Chemistry (DC3) field campaigns. In order to more easily explore this extensive dataset, an aerosol classification scheme is developed that identifies the different aerosol types measured during the deployments. This scheme makes use of ancillary data that include trace gases, chemical composition, aerodynamic particle size and geographic location, all independent of PI-Neph measurements. The PI-Neph measurements are then grouped according to their ancillary data classifications and the resulting scattering patterns are examined in detail. These results represent the first published airborne measurements of F11 and -F12/F11 for many common aerosol types. We then explore whether PI-Neph light-scattering measurements alone are sufficient to reconstruct the results of this ancillary data classification algorithm. Principal component analysis (PCA) is used to reduce the dimensionality of the multi-angle PI-Neph scattering data and the individual measurements are examined as a function of ancillary data classification. Clear clustering is observed in the PCA score space, corresponding to the ancillary classification results, suggesting that, indeed, a strong link exists between the angular-scattering measurements and the aerosol type or composition. Two techniques are used to quantify the degree of clustering and it is found that in most cases the results of the ancillary data classification can be predicted from PI-Neph measurements alone with better than 85 % recall. This result both emphasizes the validity of the ancillary data classification as well as the PI-Neph's ability to distinguish common aerosol types without additional information.

Using Information From Prior Satellite Scans to Improve Cloud Detection Near the Day-Night Terminator

NASA Technical Reports Server (NTRS)

Yost, Christopher R.; Minnis, Patrick; Trepte, Qing Z.; Palikonda, Rabindra; Ayers, Jeffrey K.; Spangenberg, Doulas A.

2012-01-01

With geostationary satellite data it is possible to have a continuous record of diurnal cycles of cloud properties for a large portion of the globe. Daytime cloud property retrieval algorithms are typically superior to nighttime algorithms because daytime methods utilize measurements of reflected solar radiation. However, reflected solar radiation is difficult to accurately model for high solar zenith angles where the amount of incident radiation is small. Clear and cloudy scenes can exhibit very small differences in reflected radiation and threshold-based cloud detection methods have more difficulty setting the proper thresholds for accurate cloud detection. Because top-of-atmosphere radiances are typically more accurately modeled outside the terminator region, information from previous scans can help guide cloud detection near the terminator. This paper presents an algorithm that uses cloud fraction and clear and cloudy infrared brightness temperatures from previous satellite scan times to improve the performance of a threshold-based cloud mask near the terminator. Comparisons of daytime, nighttime, and terminator cloud fraction derived from Geostationary Operational Environmental Satellite (GOES) radiance measurements show that the algorithm greatly reduces the number of false cloud detections and smoothes the transition from the daytime to the nighttime clod detection algorithm. Comparisons with the Geoscience Laser Altimeter System (GLAS) data show that using this algorithm decreases the number of false detections by approximately 20 percentage points.

A computer vision approach for solar radiation nowcasting using MSG images

NASA Astrophysics Data System (ADS)

Álvarez, L.; Castaño Moraga, C. A.; Martín, J.

2010-09-01

Cloud structures and haze are the two main atmospheric phenomena that reduce the performance of solar power plants, since they absorb solar energy reaching terrestrial surface. Thus, accurate forecasting of solar radiation is a challenging research area that involves both a precise localization of cloud structures and haze, as well as the attenuation introduced by these artifacts. Our work presents a novel approach for nowcasting services based on image processing techniques applied to MSG satellite images provided by the EUMETSAT Rapid Scan Service (RSS) service. These data are an interesting source of information for our purposes since every 5 minutes we obtain actual information of the atmospheric state in nearly real time. However, a workaround must be given in order to forecast solar radiation. To that end, we synthetically forecast MSG images forecasts from past images applying computer vision techniques adapted to fluid flows in order to evolve atmospheric state. First, we classify cloud structures on two different layers, corresponding to top and bottom clouds, which includes haze. This two-level classification responds to the dominant climate conditions found in our region of interest, the Canary Islands archipelago, regulated by the Gulf Stream and Trade Winds. Vertical structure of Trade Winds consists of two layers, the bottom one, which is fresh and humid, and the top one, which is warm and dry. Between these two layers a thermal inversion appears that does not allow bottom clouds to go up and naturally divides clouds in these two layers. Top clouds can be directly obtained from satellite images by means of a segmentation algorithm on histogram heights. However, bottom clouds are usually overlapped by the former, so an inpainting algorithm is used to recover overlapped areas of bottom clouds. For each layer, cloud motion is estimated through a correlation based optic flow algorithm that provides a vector field that describes the displacement field in each layer between two consecutive images in a sequence. Since RSS service from EUMETSAT provides images every 5 minutes (Δt), the cloud motion vector field between images at time t0 and (t0 - Δt) is quite similar to that between (t0 - Δt) and (t0 - 2Δt). Under this assumption, we infer the motion vector field for the next image in order to build a synthetic version of the image at time (t0 + Δt). The computation of this future motion vector field takes into account terrain orography in order to produce more realistic forecasts. In this sense, we are currently working on the integration of information from NWP outputs in order to introduce other atmospheric phenomena. Applying this algorithm several times we are able to produce short-term forecasts up to 6 hours with encouraging performance. To validate our results, we use both, comparison of synthetically generated images with the corresponding images at a given time, and direct solar radiation measurement with the set of meteorological stations located at several points of the canarian archipelago.

Detection of ground fog in mountainous areas from MODIS (Collection 051) daytime data using a statistical approach

NASA Astrophysics Data System (ADS)

Schulz, Hans Martin; Thies, Boris; Chang, Shih-Chieh; Bendix, Jörg

2016-03-01

The mountain cloud forest of Taiwan can be delimited from other forest types using a map of the ground fog frequency. In order to create such a frequency map from remotely sensed data, an algorithm able to detect ground fog is necessary. Common techniques for ground fog detection based on weather satellite data cannot be applied to fog occurrences in Taiwan as they rely on several assumptions regarding cloud properties. Therefore a new statistical method for the detection of ground fog in mountainous terrain from MODIS Collection 051 data is presented. Due to the sharpening of input data using MODIS bands 1 and 2, the method provides fog masks in a resolution of 250 m per pixel. The new technique is based on negative correlations between optical thickness and terrain height that can be observed if a cloud that is relatively plane-parallel is truncated by the terrain. A validation of the new technique using camera data has shown that the quality of fog detection is comparable to that of another modern fog detection scheme developed and validated for the temperate zones. The method is particularly applicable to optically thinner water clouds. Beyond a cloud optical thickness of ≈ 40, classification errors significantly increase.

Comparison of the filtering models for airborne LiDAR data by three classifiers with exploration on model transfer

NASA Astrophysics Data System (ADS)

Ma, Hongchao; Cai, Zhan; Zhang, Liang

2018-01-01

This paper discusses airborne light detection and ranging (LiDAR) point cloud filtering (a binary classification problem) from the machine learning point of view. We compared three supervised classifiers for point cloud filtering, namely, Adaptive Boosting, support vector machine, and random forest (RF). Nineteen features were generated from raw LiDAR point cloud based on height and other geometric information within a given neighborhood. The test datasets issued by the International Society for Photogrammetry and Remote Sensing (ISPRS) were used to evaluate the performance of the three filtering algorithms; RF showed the best results with an average total error of 5.50%. The paper also makes tentative exploration in the application of transfer learning theory to point cloud filtering, which has not been introduced into the LiDAR field to the authors' knowledge. We performed filtering of three datasets from real projects carried out in China with RF models constructed by learning from the 15 ISPRS datasets and then transferred with little to no change of the parameters. Reliable results were achieved, especially in rural area (overall accuracy achieved 95.64%), indicating the feasibility of model transfer in the context of point cloud filtering for both easy automation and acceptable accuracy.

Refinement of the CALIOP cloud mask algorithm

NASA Astrophysics Data System (ADS)

Katagiri, Shuichiro; Sato, Kaori; Ohta, Kohei; Okamoto, Hajime

2018-04-01

A modified cloud mask algorithm was applied to the CALIOP data to have more ability to detect the clouds in the lower atmosphere. In this algorithm, we also adopt the fully attenuation discrimination and the remain noise estimation using the data obtained at an altitude of 40 km to avoid contamination of stratospheric aerosols. The new cloud mask shows an increase in the lower cloud fraction. Comparison of the results to the data observed with a PML ground observation was also made.

A Ground-Based Doppler Radar and Micropulse Lidar Forward Simulator for GCM Evaluation of Arctic Mixed-Phase Clouds: Moving Forward Towards an Apples-to-apples Comparison of Hydrometeor Phase

NASA Astrophysics Data System (ADS)

Lamer, K.; Fridlind, A. M.; Ackerman, A. S.; Kollias, P.; Clothiaux, E. E.

2017-12-01

An important aspect of evaluating Artic cloud representation in a general circulation model (GCM) consists of using observational benchmarks which are as equivalent as possible to model output in order to avoid methodological bias and focus on correctly diagnosing model dynamical and microphysical misrepresentations. However, current cloud observing systems are known to suffer from biases such as limited sensitivity, and stronger response to large or small hydrometeors. Fortunately, while these observational biases cannot be corrected, they are often well understood and can be reproduced in forward simulations. Here a ground-based millimeter wavelength Doppler radar and micropulse lidar forward simulator able to interface with output from the Goddard Institute for Space Studies (GISS) ModelE GCM is presented. ModelE stratiform hydrometeor fraction, mixing ratio, mass-weighted fall speed and effective radius are forward simulated to vertically-resolved profiles of radar reflectivity, Doppler velocity and spectrum width as well as lidar backscatter and depolarization ratio. These forward simulated fields are then compared to Atmospheric Radiation Measurement (ARM) North Slope of Alaska (NSA) ground-based observations to assess cloud vertical structure (CVS). Model evalution of Arctic mixed-phase cloud would also benefit from hydrometeor phase evaluation. While phase retrieval from synergetic observations often generates large uncertainties, the same retrieval algorithm can be applied to observed and forward-simulated radar-lidar fields, thereby producing retrieved hydrometeor properties with potentially the same uncertainties. Comparing hydrometeor properties retrieved in exactly the same way aims to produce the best apples-to-apples comparisons between GCM ouputs and observations. The use of a comprenhensive ground-based forward simulator coupled with a hydrometeor classification retrieval algorithm provides a new perspective for GCM evaluation of Arctic mixed-phase clouds from the ground where low-level supercooled liquid layer are more easily observed and where additional environmental properties such as cloud condensation nuclei are quantified. This should help assist in choosing between several possible diagnostic ice nucleation schemes for ModelE stratiform cloud.

Evaluation of Passive Multilayer Cloud Detection Using Preliminary CloudSat and CALIPSO Cloud Profiles

NASA Astrophysics Data System (ADS)

Minnis, P.; Sun-Mack, S.; Chang, F.; Huang, J.; Nguyen, L.; Ayers, J. K.; Spangenberg, D. A.; Yi, Y.; Trepte, C. R.

2006-12-01

During the last few years, several algorithms have been developed to detect and retrieve multilayered clouds using passive satellite data. Assessing these techniques has been difficult due to the need for active sensors such as cloud radars and lidars that can "see" through different layers of clouds. Such sensors have been available only at a few surface sites and on aircraft during field programs. With the launch of the CALIPSO and CloudSat satellites on April 28, 2006, it is now possible to observe multilayered systems all over the globe using collocated cloud radar and lidar data. As part of the A- Train, these new active sensors are also matched in time ad space with passive measurements from the Aqua Moderate Resolution Imaging Spectroradiometer (MODIS) and Advanced Microwave Scanning Radiometer - EOS (AMSR-E). The Clouds and the Earth's Radiant Energy System (CERES) has been developing and testing algorithms to detect ice-over-water overlapping cloud systems and to retrieve the cloud liquid path (LWP) and ice water path (IWP) for those systems. One technique uses a combination of the CERES cloud retrieval algorithm applied to MODIS data and a microwave retrieval method applied to AMSR-E data. The combination of a CO2-slicing cloud retireval technique with the CERES algorithms applied to MODIS data (Chang et al., 2005) is used to detect and analyze such overlapped systems that contain thin ice clouds. A third technique uses brightness temperature differences and the CERES algorithms to detect similar overlapped methods. This paper uses preliminary CloudSat and CALIPSO data to begin a global scale assessment of these different methods. The long-term goals are to assess and refine the algorithms to aid the development of an optimal combination of the techniques to better monitor ice 9and liquid water clouds in overlapped conditions.

Evaluation of Passive Multilayer Cloud Detection Using Preliminary CloudSat and CALIPSO Cloud Profiles

NASA Astrophysics Data System (ADS)

Minnis, P.; Sun-Mack, S.; Chang, F.; Huang, J.; Nguyen, L.; Ayers, J. K.; Spangenberg, D. A.; Yi, Y.; Trepte, C. R.

2005-05-01

During the last few years, several algorithms have been developed to detect and retrieve multilayered clouds using passive satellite data. Assessing these techniques has been difficult due to the need for active sensors such as cloud radars and lidars that can "see" through different layers of clouds. Such sensors have been available only at a few surface sites and on aircraft during field programs. With the launch of the CALIPSO and CloudSat satellites on April 28, 2006, it is now possible to observe multilayered systems all over the globe using collocated cloud radar and lidar data. As part of the A- Train, these new active sensors are also matched in time ad space with passive measurements from the Aqua Moderate Resolution Imaging Spectroradiometer (MODIS) and Advanced Microwave Scanning Radiometer - EOS (AMSR-E). The Clouds and the Earth's Radiant Energy System (CERES) has been developing and testing algorithms to detect ice-over-water overlapping cloud systems and to retrieve the cloud liquid path (LWP) and ice water path (IWP) for those systems. One technique uses a combination of the CERES cloud retrieval algorithm applied to MODIS data and a microwave retrieval method applied to AMSR-E data. The combination of a CO2-slicing cloud retireval technique with the CERES algorithms applied to MODIS data (Chang et al., 2005) is used to detect and analyze such overlapped systems that contain thin ice clouds. A third technique uses brightness temperature differences and the CERES algorithms to detect similar overlapped methods. This paper uses preliminary CloudSat and CALIPSO data to begin a global scale assessment of these different methods. The long-term goals are to assess and refine the algorithms to aid the development of an optimal combination of the techniques to better monitor ice 9and liquid water clouds in overlapped conditions.

A 3D Cloud-Construction Algorithm for the EarthCARE Satellite Mission

NASA Technical Reports Server (NTRS)

Barker, H. W.; Jerg, M. P.; Wehr, T.; Kato, S.; Donovan, D. P.; Hogan, R. J.

2011-01-01

This article presents and assesses an algorithm that constructs 3D distributions of cloud from passive satellite imagery and collocated 2D nadir profiles of cloud properties inferred synergistically from lidar, cloud radar and imager data.

Spatial characteristics of the tropical cloud systems: comparison between model simulation and satellite observations

NASA Astrophysics Data System (ADS)

Zhang, Guang J.; Zurovac-Jevtic, Dance; Boer, Erwin R.

1999-10-01

A Lagrangian cloud classification algorithm is applied to the cloud fields in the tropical Pacific simulated by a high-resolution regional atmospheric model. The purpose of this work is to assess the model's ability to reproduce the observed spatial characteristics of the tropical cloud systems. The cloud systems are broadly grouped into three categories: deep clouds, mid-level clouds and low clouds. The deep clouds are further divided into mesoscale convective systems and nonmesoscale convective systems. It is shown that the model is able to simulate the total cloud cover for each category reasonably well. However, when the cloud cover is broken down into contributions from cloud systems of different sizes, it is shown that the simulated cloud size distribution is biased toward large cloud systems, with contribution from relatively small cloud systems significantly under-represented in the model for both deep and mid-level clouds. The number distribution and area contribution to the cloud cover from mesoscale convective systems are very well simulated compared to the satellite observations, so are low clouds as well. The dependence of the cloud physical properties on cloud scale is examined. It is found that cloud liquid water path, rainfall, and ocean surface sensible and latent heat fluxes have a clear dependence on cloud types and scale. This is of particular interest to studies of the cloud effects on surface energy budget and hydrological cycle. The diurnal variation of the cloud population and area is also examined. The model exhibits a varying degree of success in simulating the diurnal variation of the cloud number and area. The observed early morning maximum cloud cover in deep convective cloud systems is qualitatively simulated. However, the afternoon secondary maximum is missing in the model simulation. The diurnal variation of the tropospheric temperature is well reproduced by the model while simulation of the diurnal variation of the moisture field is poor. The implication of this comparison between model simulation and observations on cloud parameterization is discussed.

Large Scale Crop Mapping in Ukraine Using Google Earth Engine

NASA Astrophysics Data System (ADS)

Shelestov, A.; Lavreniuk, M. S.; Kussul, N.

2016-12-01

There are no globally available high resolution satellite-derived crop specific maps at present. Only coarse-resolution imagery (> 250 m spatial resolution) has been utilized to derive global cropland extent. In 2016 we are going to carry out a country level demonstration of Sentinel-2 use for crop classification in Ukraine within the ESA Sen2-Agri project. But optical imagery can be contaminated by cloud cover that makes it difficult to acquire imagery in an optimal time range to discriminate certain crops. Due to the Copernicus program since 2015, a lot of Sentinel-1 SAR data at high spatial resolution is available for free for Ukraine. It allows us to use the time series of SAR data for crop classification. Our experiment for one administrative region in 2015 showed much higher crop classification accuracy with SAR data than with optical only time series [1, 2]. Therefore, in 2016 within the Google Earth Engine Research Award we use SAR data together with optical ones for large area crop mapping (entire territory of Ukraine) using cloud computing capabilities available at Google Earth Engine (GEE). This study compares different classification methods for crop mapping for the whole territory of Ukraine using data and algorithms from GEE. Classification performance assessed using overall classification accuracy, Kappa coefficients, and user's and producer's accuracies. Also, crop areas from derived classification maps compared to the official statistics [3]. S. Skakun et al., "Efficiency assessment of multitemporal C-band Radarsat-2 intensity and Landsat-8 surface reflectance satellite imagery for crop classification in Ukraine," IEEE Journal of Selected Topics in Applied Earth Observ. and Rem. Sens., 2015, DOI: 10.1109/JSTARS.2015.2454297. N. Kussul, S. Skakun, A. Shelestov, O. Kussul, "The use of satellite SAR imagery to crop classification in Ukraine within JECAM project," IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp.1497-1500, 13-18 July 2014, Quebec City, Canada. F.J. Gallego, N. Kussul, S. Skakun, O. Kravchenko, A. Shelestov, O. Kussul, "Efficiency assessment of using satellite data for crop area estimation in Ukraine," International Journal of Applied Earth Observation and Geoinformation vol. 29, pp. 22-30, 2014.

An Improved Algorithm for Retrieving Surface Downwelling Longwave Radiation from Satellite Measurements

NASA Technical Reports Server (NTRS)

Zhou, Yaping; Kratz, David P.; Wilber, Anne C.; Gupta, Shashi K.; Cess, Robert D.

2006-01-01

Retrieving surface longwave radiation from space has been a difficult task since the surface downwelling longwave radiation (SDLW) are integrations from radiation emitted by the entire atmosphere, while those emitted from the upper atmosphere are absorbed before reaching the surface. It is particularly problematic when thick clouds are present since thick clouds will virtually block all the longwave radiation from above, while satellites observe atmosphere emissions mostly from above the clouds. Zhou and Cess developed an algorithm for retrieving SDLW based upon detailed studies using radiative transfer model calculations and surface radiometric measurements. Their algorithm linked clear sky SDLW with surface upwelling longwave flux and column precipitable water vapor. For cloudy sky cases, they used cloud liquid water path as an additional parameter to account for the effects of clouds. Despite the simplicity of their algorithm, it performed very well for most geographical regions except for those regions where the atmospheric conditions near the surface tend to be extremely cold and dry. Systematic errors were also found for areas that were covered with ice clouds. An improved version of the algorithm was developed that prevents the large errors in the SDLW at low water vapor amounts. The new algorithm also utilizes cloud fraction and cloud liquid and ice water paths measured from the Cloud and the Earth's Radiant Energy System (CERES) satellites to separately compute the clear and cloudy portions of the fluxes. The new algorithm has been validated against surface measurements at 29 stations around the globe for the Terra and Aqua satellites. The results show significant improvement over the original version. The revised Zhou-Cess algorithm is also slightly better or comparable to more sophisticated algorithms currently implemented in the CERES processing. It will be incorporated in the CERES project as one of the empirical surface radiation algorithms.

Improvements in Night-Time Low Cloud Detection and MODIS-Style Cloud Optical Properties from MSG SEVIRI

NASA Technical Reports Server (NTRS)

Wind, Galina (Gala); Platnick, Steven; Riedi, Jerome

2011-01-01

The MODIS cloud optical properties algorithm (MOD06IMYD06 for Terra and Aqua MODIS, respectively) slated for production in Data Collection 6 has been adapted to execute using available channels on MSG SEVIRI. Available MODIS-style retrievals include IR Window-derived cloud top properties, using the new Collection 6 cloud top properties algorithm, cloud optical thickness from VISINIR bands, cloud effective radius from 1.6 and 3.7Jlm and cloud ice/water path. We also provide pixel-level uncertainty estimate for successful retrievals. It was found that at nighttime the SEVIRI cloud mask tends to report unnaturally low cloud fraction for marine stratocumulus clouds. A correction algorithm that improves detection of such clouds has been developed. We will discuss the improvements to nighttime low cloud detection for SEVIRI and show examples and comparisons with MODIS and CALIPSO. We will also show examples of MODIS-style pixel-level (Level-2) cloud retrievals for SEVIRI with comparisons to MODIS.

Retrieving Atmospheric Profiles Data in the Presence of Clouds from Hyperspectral Remote Sensing Data

NASA Technical Reports Server (NTRS)

Liu, Xu; Larar, Allen M.; Zhou, Daniel K.; Kizer, Susan H.; Wu, Wan; Barnet, Christopher; Divakarla, Murty; Guo, Guang; Blackwell, Bill; Smith, William L.;

Automatic Cloud Detection from Multi-Temporal Satellite Images: Towards the Use of PLÉIADES Time Series

NASA Astrophysics Data System (ADS)

Champion, N.

2012-08-01

Contrary to aerial images, satellite images are often affected by the presence of clouds. Identifying and removing these clouds is one of the primary steps to perform when processing satellite images, as they may alter subsequent procedures such as atmospheric corrections, DSM production or land cover classification. The main goal of this paper is to present the cloud detection approach, developed at the French Mapping agency. Our approach is based on the availability of multi-temporal satellite images (i.e. time series that generally contain between 5 and 10 images) and is based on a region-growing procedure. Seeds (corresponding to clouds) are firstly extracted through a pixel-to-pixel comparison between the images contained in time series (the presence of a cloud is here assumed to be related to a high variation of reflectance between two images). Clouds are then delineated finely using a dedicated region-growing algorithm. The method, originally designed for panchromatic SPOT5-HRS images, is tested in this paper using time series with 9 multi-temporal satellite images. Our preliminary experiments show the good performances of our method. In a near future, the method will be applied to Pléiades images, acquired during the in-flight commissioning phase of the satellite (launched at the end of 2011). In that context, this is a particular goal of this paper to show to which extent and in which way our method can be adapted to this kind of imagery.

Fully Convolutional Networks for Ground Classification from LIDAR Point Clouds

NASA Astrophysics Data System (ADS)

Rizaldy, A.; Persello, C.; Gevaert, C. M.; Oude Elberink, S. J.

2018-05-01

Deep Learning has been massively used for image classification in recent years. The use of deep learning for ground classification from LIDAR point clouds has also been recently studied. However, point clouds need to be converted into an image in order to use Convolutional Neural Networks (CNNs). In state-of-the-art techniques, this conversion is slow because each point is converted into a separate image. This approach leads to highly redundant computation during conversion and classification. The goal of this study is to design a more efficient data conversion and ground classification. This goal is achieved by first converting the whole point cloud into a single image. The classification is then performed by a Fully Convolutional Network (FCN), a modified version of CNN designed for pixel-wise image classification. The proposed method is significantly faster than state-of-the-art techniques. On the ISPRS Filter Test dataset, it is 78 times faster for conversion and 16 times faster for classification. Our experimental analysis on the same dataset shows that the proposed method results in 5.22 % of total error, 4.10 % of type I error, and 15.07 % of type II error. Compared to the previous CNN-based technique and LAStools software, the proposed method reduces the total error and type I error (while type II error is slightly higher). The method was also tested on a very high point density LIDAR point clouds resulting in 4.02 % of total error, 2.15 % of type I error and 6.14 % of type II error.

D Land Cover Classification Based on Multispectral LIDAR Point Clouds

NASA Astrophysics Data System (ADS)

Zou, Xiaoliang; Zhao, Guihua; Li, Jonathan; Yang, Yuanxi; Fang, Yong

2016-06-01

Multispectral Lidar System can emit simultaneous laser pulses at the different wavelengths. The reflected multispectral energy is captured through a receiver of the sensor, and the return signal together with the position and orientation information of sensor is recorded. These recorded data are solved with GNSS/IMU data for further post-processing, forming high density multispectral 3D point clouds. As the first commercial multispectral airborne Lidar sensor, Optech Titan system is capable of collecting point clouds data from all three channels at 532nm visible (Green), at 1064 nm near infrared (NIR) and at 1550nm intermediate infrared (IR). It has become a new source of data for 3D land cover classification. The paper presents an Object Based Image Analysis (OBIA) approach to only use multispectral Lidar point clouds datasets for 3D land cover classification. The approach consists of three steps. Firstly, multispectral intensity images are segmented into image objects on the basis of multi-resolution segmentation integrating different scale parameters. Secondly, intensity objects are classified into nine categories by using the customized features of classification indexes and a combination the multispectral reflectance with the vertical distribution of object features. Finally, accuracy assessment is conducted via comparing random reference samples points from google imagery tiles with the classification results. The classification results show higher overall accuracy for most of the land cover types. Over 90% of overall accuracy is achieved via using multispectral Lidar point clouds for 3D land cover classification.

Improving Satellite Quantitative Precipitation Estimation Using GOES-Retrieved Cloud Optical Depth

DOE Office of Scientific and Technical Information (OSTI.GOV)

Stenz, Ronald; Dong, Xiquan; Xi, Baike

To address significant gaps in ground-based radar coverage and rain gauge networks in the U.S., geostationary satellite quantitative precipitation estimates (QPEs) such as the Self-Calibrating Multivariate Precipitation Retrievals (SCaMPR) can be used to fill in both the spatial and temporal gaps of ground-based measurements. Additionally, with the launch of GOES-R, the temporal resolution of satellite QPEs may be comparable to that of Weather Service Radar-1988 Doppler (WSR-88D) volume scans as GOES images will be available every five minutes. However, while satellite QPEs have strengths in spatial coverage and temporal resolution, they face limitations particularly during convective events. Deep Convective Systemsmore » (DCSs) have large cloud shields with similar brightness temperatures (BTs) over nearly the entire system, but widely varying precipitation rates beneath these clouds. Geostationary satellite QPEs relying on the indirect relationship between BTs and precipitation rates often suffer from large errors because anvil regions (little/no precipitation) cannot be distinguished from rain-cores (heavy precipitation) using only BTs. However, a combination of BTs and optical depth (τ) has been found to reduce overestimates of precipitation in anvil regions (Stenz et al. 2014). A new rain mask algorithm incorporating both τ and BTs has been developed, and its application to the existing SCaMPR algorithm was evaluated. The performance of the modified SCaMPR was evaluated using traditional skill scores and a more detailed analysis of performance in individual DCS components by utilizing the Feng et al. (2012) classification algorithm. SCaMPR estimates with the new rain mask applied benefited from significantly reduced overestimates of precipitation in anvil regions and overall improvements in skill scores.« less

Satellite Data Analysis of Impact of Anthropogenic Air Pollution on Ice Clouds

NASA Astrophysics Data System (ADS)

Gu, Y.; Liou, K. N.; Zhao, B.; Jiang, J. H.; Su, H.

2017-12-01

Despite numerous studies about the impact of aerosols on ice clouds, the role of anthropogenic aerosols in ice processes, especially over pollution regions, remains unclear and controversial, and has not been considered in a regional model. The objective of this study is to improve our understanding of the ice process associated with anthropogenic aerosols, and provide a comprehensive assessment of the contribution of anthropogenic aerosols to ice nucleation, ice cloud properties, and the consequent regional radiative forcing. As the first attempt, we evaluate the effects of different aerosol types (mineral dust, air pollution, polluted dust, and smoke) on ice cloud micro- and macro-physical properties using satellite data. We identify cases with collocated CloudSat, CALIPSO, and Aqua observations of vertically resolved aerosol and cloud properties, and process these observations into the same spatial resolution. The CALIPSO's aerosol classification algorithm determines aerosol layers as one of six defined aerosol types by taking into account the lidar depolarization ratio, integrated attenuated backscattering, surface type, and layer elevation. We categorize the cases identified above according to aerosol types, collect relevant aerosol and ice cloud variables, and determine the correlation between column/layer AOD and ice cloud properties for each aerosol type. Specifically, we investigate the correlation between aerosol loading (indicated by the column AOD and layer AOD) and ice cloud microphysical properties (ice water content, ice crystal number concentration, and ice crystal effective radius) and macro-physical properties (ice water path, ice cloud fraction, cloud top temperature, and cloud thickness). By comparing the responses of ice cloud properties to aerosol loadings for different aerosol types, we infer the role of different aerosol types in ice nucleation and the evolution of ice clouds. Our preliminary study shows that changes in the ice crystal effective radius with respect to AOD over Eastern Asia for the aerosol types of polluted continental and mineral dust look similar, implying that both air pollution and mineral dust could affect the microphysical properties of ice clouds.

A Distributed Parallel Genetic Algorithm of Placement Strategy for Virtual Machines Deployment on Cloud Platform

PubMed Central

Dong, Yu-Shuang; Xu, Gao-Chao; Fu, Xiao-Dong

2014-01-01

The cloud platform provides various services to users. More and more cloud centers provide infrastructure as the main way of operating. To improve the utilization rate of the cloud center and to decrease the operating cost, the cloud center provides services according to requirements of users by sharding the resources with virtualization. Considering both QoS for users and cost saving for cloud computing providers, we try to maximize performance and minimize energy cost as well. In this paper, we propose a distributed parallel genetic algorithm (DPGA) of placement strategy for virtual machines deployment on cloud platform. It executes the genetic algorithm parallelly and distributedly on several selected physical hosts in the first stage. Then it continues to execute the genetic algorithm of the second stage with solutions obtained from the first stage as the initial population. The solution calculated by the genetic algorithm of the second stage is the optimal one of the proposed approach. The experimental results show that the proposed placement strategy of VM deployment can ensure QoS for users and it is more effective and more energy efficient than other placement strategies on the cloud platform. PMID:25097872

A distributed parallel genetic algorithm of placement strategy for virtual machines deployment on cloud platform.

PubMed

Dong, Yu-Shuang; Xu, Gao-Chao; Fu, Xiao-Dong

2014-01-01

The cloud platform provides various services to users. More and more cloud centers provide infrastructure as the main way of operating. To improve the utilization rate of the cloud center and to decrease the operating cost, the cloud center provides services according to requirements of users by sharding the resources with virtualization. Considering both QoS for users and cost saving for cloud computing providers, we try to maximize performance and minimize energy cost as well. In this paper, we propose a distributed parallel genetic algorithm (DPGA) of placement strategy for virtual machines deployment on cloud platform. It executes the genetic algorithm parallelly and distributedly on several selected physical hosts in the first stage. Then it continues to execute the genetic algorithm of the second stage with solutions obtained from the first stage as the initial population. The solution calculated by the genetic algorithm of the second stage is the optimal one of the proposed approach. The experimental results show that the proposed placement strategy of VM deployment can ensure QoS for users and it is more effective and more energy efficient than other placement strategies on the cloud platform.

FPGA Coprocessor for Accelerated Classification of Images

NASA Technical Reports Server (NTRS)

Pingree, Paula J.; Scharenbroich, Lucas J.; Werne, Thomas A.

2008-01-01

An effort related to that described in the preceding article focuses on developing a spaceborne processing platform for fast and accurate onboard classification of image data, a critical part of modern satellite image processing. The approach again has been to exploit the versatility of recently developed hybrid Virtex-4FX field-programmable gate array (FPGA) to run diverse science applications on embedded processors while taking advantage of the reconfigurable hardware resources of the FPGAs. In this case, the FPGA serves as a coprocessor that implements legacy C-language support-vector-machine (SVM) image-classification algorithms to detect and identify natural phenomena such as flooding, volcanic eruptions, and sea-ice break-up. The FPGA provides hardware acceleration for increased onboard processing capability than previously demonstrated in software. The original C-language program demonstrated on an imaging instrument aboard the Earth Observing-1 (EO-1) satellite implements a linear-kernel SVM algorithm for classifying parts of the images as snow, water, ice, land, or cloud or unclassified. Current onboard processors, such as on EO-1, have limited computing power, extremely limited active storage capability and are no longer considered state-of-the-art. Using commercially available software that translates C-language programs into hardware description language (HDL) files, the legacy C-language program, and two newly formulated programs for a more capable expanded-linear-kernel and a more accurate polynomial-kernel SVM algorithm, have been implemented in the Virtex-4FX FPGA. In tests, the FPGA implementations have exhibited significant speedups over conventional software implementations running on general-purpose hardware.

A New Algorithm for Detecting Cloud Height using OMPS/LP Measurements

NASA Technical Reports Server (NTRS)

Chen, Zhong; DeLand, Matthew; Bhartia, Pawan K.

2016-01-01

The Ozone Mapping and Profiler Suite Limb Profiler (OMPS/LP) ozone product requires the determination of cloud height for each event to establish the lower boundary of the profile for the retrieval algorithm. We have created a revised cloud detection algorithm for LP measurements that uses the spectral dependence of the vertical gradient in radiance between two wavelengths in the visible and near-IR spectral regions. This approach provides better discrimination between clouds and aerosols than results obtained using a single wavelength. Observed LP cloud height values show good agreement with coincident Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) measurements.

Job Scheduling with Efficient Resource Monitoring in Cloud Datacenter

PubMed Central

Loganathan, Shyamala; Mukherjee, Saswati

2015-01-01

Cloud computing is an on-demand computing model, which uses virtualization technology to provide cloud resources to users in the form of virtual machines through internet. Being an adaptable technology, cloud computing is an excellent alternative for organizations for forming their own private cloud. Since the resources are limited in these private clouds maximizing the utilization of resources and giving the guaranteed service for the user are the ultimate goal. For that, efficient scheduling is needed. This research reports on an efficient data structure for resource management and resource scheduling technique in a private cloud environment and discusses a cloud model. The proposed scheduling algorithm considers the types of jobs and the resource availability in its scheduling decision. Finally, we conducted simulations using CloudSim and compared our algorithm with other existing methods, like V-MCT and priority scheduling algorithms. PMID:26473166

Job Scheduling with Efficient Resource Monitoring in Cloud Datacenter.

PubMed

Loganathan, Shyamala; Mukherjee, Saswati

2015-01-01

Cloud computing is an on-demand computing model, which uses virtualization technology to provide cloud resources to users in the form of virtual machines through internet. Being an adaptable technology, cloud computing is an excellent alternative for organizations for forming their own private cloud. Since the resources are limited in these private clouds maximizing the utilization of resources and giving the guaranteed service for the user are the ultimate goal. For that, efficient scheduling is needed. This research reports on an efficient data structure for resource management and resource scheduling technique in a private cloud environment and discusses a cloud model. The proposed scheduling algorithm considers the types of jobs and the resource availability in its scheduling decision. Finally, we conducted simulations using CloudSim and compared our algorithm with other existing methods, like V-MCT and priority scheduling algorithms.

A mangrove forest map of China in 2015: Analysis of time series Landsat 7/8 and Sentinel-1A imagery in Google Earth Engine cloud computing platform

NASA Astrophysics Data System (ADS)

Chen, Bangqian; Xiao, Xiangming; Li, Xiangping; Pan, Lianghao; Doughty, Russell; Ma, Jun; Dong, Jinwei; Qin, Yuanwei; Zhao, Bin; Wu, Zhixiang; Sun, Rui; Lan, Guoyu; Xie, Guishui; Clinton, Nicholas; Giri, Chandra

2017-09-01

Due to rapid losses of mangrove forests caused by anthropogenic disturbances and climate change, accurate and contemporary maps of mangrove forests are needed to understand how mangrove ecosystems are changing and establish plans for sustainable management. In this study, a new classification algorithm was developed using the biophysical characteristics of mangrove forests in China. More specifically, these forests were mapped by identifying: (1) greenness, canopy coverage, and tidal inundation from time series Landsat data, and (2) elevation, slope, and intersection-with-sea criterion. The annual mean Normalized Difference Vegetation Index (NDVI) was found to be a key variable in determining the classification thresholds of greenness, canopy coverage, and tidal inundation of mangrove forests, which are greatly affected by tide dynamics. In addition, the integration of Sentinel-1A VH band and modified Normalized Difference Water Index (mNDWI) shows great potential in identifying yearlong tidal and fresh water bodies, which is related to mangrove forests. This algorithm was developed using 6 typical Regions of Interest (ROIs) as algorithm training and was run on the Google Earth Engine (GEE) cloud computing platform to process 1941 Landsat images (25 Path/Row) and 586 Sentinel-1A images circa 2015. The resultant mangrove forest map of China at 30 m spatial resolution has an overall/users/producer's accuracy greater than 95% when validated with ground reference data. In 2015, China's mangrove forests had a total area of 20,303 ha, about 92% of which was in the Guangxi Zhuang Autonomous Region, Guangdong, and Hainan Provinces. This study has demonstrated the potential of using the GEE platform, time series Landsat and Sentine-1A SAR images to identify and map mangrove forests along the coastal zones. The resultant mangrove forest maps are likely to be useful for the sustainable management and ecological assessments of mangrove forests in China.

Global velocity constrained cloud motion prediction for short-term solar forecasting

NASA Astrophysics Data System (ADS)

Chen, Yanjun; Li, Wei; Zhang, Chongyang; Hu, Chuanping

2016-09-01

Cloud motion is the primary reason for short-term solar power output fluctuation. In this work, a new cloud motion estimation algorithm using a global velocity constraint is proposed. Compared to the most used Particle Image Velocity (PIV) algorithm, which assumes the homogeneity of motion vectors, the proposed method can capture the accurate motion vector for each cloud block, including both the motional tendency and morphological changes. Specifically, global velocity derived from PIV is first calculated, and then fine-grained cloud motion estimation can be achieved by global velocity based cloud block researching and multi-scale cloud block matching. Experimental results show that the proposed global velocity constrained cloud motion prediction achieves comparable performance to the existing PIV and filtered PIV algorithms, especially in a short prediction horizon.

A Deep Machine Learning Algorithm to Optimize the Forecast of Atmospherics

NASA Astrophysics Data System (ADS)

Russell, A. M.; Alliss, R. J.; Felton, B. D.

Space-based applications from imaging to optical communications are significantly impacted by the atmosphere. Specifically, the occurrence of clouds and optical turbulence can determine whether a mission is a success or a failure. In the case of space-based imaging applications, clouds produce atmospheric transmission losses that can make it impossible for an electro-optical platform to image its target. Hence, accurate predictions of negative atmospheric effects are a high priority in order to facilitate the efficient scheduling of resources. This study seeks to revolutionize our understanding of and our ability to predict such atmospheric events through the mining of data from a high-resolution Numerical Weather Prediction (NWP) model. Specifically, output from the Weather Research and Forecasting (WRF) model is mined using a Random Forest (RF) ensemble classification and regression approach in order to improve the prediction of low cloud cover over the Haleakala summit of the Hawaiian island of Maui. RF techniques have a number of advantages including the ability to capture non-linear associations between the predictors (in this case physical variables from WRF such as temperature, relative humidity, wind speed and pressure) and the predictand (clouds), which becomes critical when dealing with the complex non-linear occurrence of clouds. In addition, RF techniques are capable of representing complex spatial-temporal dynamics to some extent. Input predictors to the WRF-based RF model are strategically selected based on expert knowledge and a series of sensitivity tests. Ultimately, three types of WRF predictors are chosen: local surface predictors, regional 3D moisture predictors and regional inversion predictors. A suite of RF experiments is performed using these predictors in order to evaluate the performance of the hybrid RF-WRF technique. The RF model is trained and tuned on approximately half of the input dataset and evaluated on the other half. The RF approach is validated using in-situ observations of clouds. All of the hybrid RF-WRF experiments demonstrated here significantly outperform the base WRF local low cloud cover forecasts in terms of the probability of detection and the overall bias. In particular, RF experiments that use only regional three-dimensional moisture predictors from the WRF model produce the highest accuracy when compared to RF experiments that use local surface predictors only or regional inversion predictors only. Furthermore, adding multiple types of WRF predictors and additional WRF predictors to the RF algorithm does not necessarily add more value in the resulting forecasts, indicating that it is better to have a small set of meaningful predictors than to have a vast set of indiscriminately-chosen predictors. This work also reveals that the WRF-based RF approach is highly sensitive to the time period over which the algorithm is trained and evaluated. Future work will focus on developing a similar WRF-based RF model for high cloud prediction and expanding the algorithm to two-dimensions horizontally.

Fast Simulators for Satellite Cloud Optical Centroid Pressure Retrievals, 1. Evaluation of OMI Cloud Retrievals

NASA Technical Reports Server (NTRS)

Joiner, J.; Vasilkov, A. P.; Gupta, Pawan; Bhartia, P. K.; Veefkind, Pepijn; Sneep, Maarten; deHaan, Johan; Polonsky, Igor; Spurr, Robert

2011-01-01

We have developed a relatively simple scheme for simulating retrieved cloud optical centroid pressures (OCP) from satellite solar backscatter observations. We have compared simulator results with those from more detailed retrieval simulators that more fully account for the complex radiative transfer in a cloudy atmosphere. We used this fast simulator to conduct a comprehensive evaluation of cloud OCPs from the two OMI algorithms using collocated data from CloudSat and Aqua MODIS, a unique situation afforded by the A-train formation of satellites. We find that both OMI algorithms perform reasonably well and that the two algorithms agree better with each other than either does with the collocated CloudSat data. This indicates that patchy snow/ice, cloud 3D, and aerosol effects not simulated with the CloudSat data are affecting both algorithms similarly. We note that the collocation with CloudSat occurs mainly on the East side of OMI's swath. Therefore, we are not able to address cross-track biases in OMI cloud OCP retrievals. Our fast simulator may also be used to simulate cloud OCP from output generated by general circulation models (GCM) with appropriate account of cloud overlap. We have implemented such a scheme and plan to compare OMI data with GCM output in the near future.

Development of GK-2A cloud optical and microphysical properties retrieval algorithm

NASA Astrophysics Data System (ADS)

Yang, Y.; Yum, S. S.; Um, J.

2017-12-01

Cloud and aerosol radiative forcing is known to be one of the the largest uncertainties in climate change prediction. To reduce this uncertainty, remote sensing observation of cloud radiative and microphysical properties have been used since 1970s and the corresponding remote sensing techniques and instruments have been developed. As a part of such effort, Geo-KOMPSAT-2A (Geostationary Korea Multi-Purpose Satellite-2A, GK-2A) will be launched in 2018. On the GK-2A, the Advanced Meteorological Imager (AMI) is primary instrument which have 3 visible, 3 near-infrared, and 10 infrared channels. To retrieve optical and microphysical properties of clouds using AMI measurements, the preliminary version of new cloud retrieval algorithm for GK-2A was developed and several validation tests were conducted. This algorithm retrieves cloud optical thickness (COT), cloud effective radius (CER), liquid water path (LWP), and ice water path (IWP), so we named this algorithm as Daytime Cloud Optical thickness, Effective radius and liquid and ice Water path (DCOEW). The DCOEW uses cloud reflectance at visible and near-infrared channels as input data. An optimal estimation (OE) approach that requires appropriate a-priori values and measurement error information is used to retrieve COT and CER. LWP and IWP are calculated using empirical relationships between COT/CER and cloud water path that were determined previously. To validate retrieved cloud properties, we compared DCOEW output data with other operational satellite data. For COT and CER validation, we used two different data sets. To compare algorithms that use cloud reflectance at visible and near-IR channels as input data, MODIS MYD06 cloud product was selected. For the validation with cloud products that are based on microwave measurements, COT(2B-TAU)/CER(2C-ICE) data retrieved from CloudSat cloud profiling radar (W-band, 94 GHz) was used. For cloud water path validation, AMSR-2 Level-3 Cloud liquid water data was used. Detailed results will be shown at the conference.

Cloud cover determination in polar regions from satellite imagery

NASA Technical Reports Server (NTRS)

Barry, R. G.; Key, J.

1989-01-01

The objectives are to develop a suitable validation data set for evaluating the effectiveness of the International Satellite Cloud Climatology Project (ISCCP) algorithm for cloud retrieval in polar regions, to identify limitations of current procedures and to explore potential means to remedy them using textural classifiers, and to compare synoptic cloud data from model runs with observations. Toward the first goal, a polar data set consisting of visible, thermal, and passive microwave data was developed. The AVHRR and SMMR data were digitally merged to a polar stereographic projection with an effective pixel size of 5 sq km. With this data set, two unconventional methods of classifying the imagery for the analysis of polar clouds and surfaces were examined: one based on fuzzy sets theory and another based on a trained neural network. An algorithm for cloud detection was developed from an early test version of the ISCCP algorithm. This algorithm includes the identification of surface types with passive microwave, then temporal tests at each pixel location in the cloud detection phase. Cloud maps and clear sky radiance composites for 5 day periods are produced. Algorithm testing and validation was done with both actural AVHRR/SMMR data, and simulated imagery. From this point in the algorithm, groups of cloud pixels are examined for their spectral and textural characteristics, and a procedure is developed for the analysis of cloud patterns utilizing albedo, IR temperature, and texture. In a completion of earlier work, empirical analyses of arctic cloud cover were explored through manual interpretations of DMSP imagery and compared to U.S. Air Force 3D-nephanalysis. Comparisons of observed cloudiness from existing climatologies to patterns computed by the GISS climate model were also made.

Automatic extraction of blocks from 3D point clouds of fractured rock

NASA Astrophysics Data System (ADS)

Chen, Na; Kemeny, John; Jiang, Qinghui; Pan, Zhiwen

2017-12-01

This paper presents a new method for extracting blocks and calculating block size automatically from rock surface 3D point clouds. Block size is an important rock mass characteristic and forms the basis for several rock mass classification schemes. The proposed method consists of four steps: 1) the automatic extraction of discontinuities using an improved Ransac Shape Detection method, 2) the calculation of discontinuity intersections based on plane geometry, 3) the extraction of block candidates based on three discontinuities intersecting one another to form corners, and 4) the identification of "true" blocks using an improved Floodfill algorithm. The calculated block sizes were compared with manual measurements in two case studies, one with fabricated cardboard blocks and the other from an actual rock mass outcrop. The results demonstrate that the proposed method is accurate and overcomes the inaccuracies, safety hazards, and biases of traditional techniques.

A clinical decision-making mechanism for context-aware and patient-specific remote monitoring systems using the correlations of multiple vital signs.

PubMed

Forkan, Abdur Rahim Mohammad; Khalil, Ibrahim

2017-02-01

In home-based context-aware monitoring patient's real-time data of multiple vital signs (e.g. heart rate, blood pressure) are continuously generated from wearable sensors. The changes in such vital parameters are highly correlated. They are also patient-centric and can be either recurrent or can fluctuate. The objective of this study is to develop an intelligent method for personalized monitoring and clinical decision support through early estimation of patient-specific vital sign values, and prediction of anomalies using the interrelation among multiple vital signs. In this paper, multi-label classification algorithms are applied in classifier design to forecast these values and related abnormalities. We proposed a completely new approach of patient-specific vital sign prediction system using their correlations. The developed technique can guide healthcare professionals to make accurate clinical decisions. Moreover, our model can support many patients with various clinical conditions concurrently by utilizing the power of cloud computing technology. The developed method also reduces the rate of false predictions in remote monitoring centres. In the experimental settings, the statistical features and correlations of six vital signs are formulated as multi-label classification problem. Eight multi-label classification algorithms along with three fundamental machine learning algorithms are used and tested on a public dataset of 85 patients. Different multi-label classification evaluation measures such as Hamming score, F1-micro average, and accuracy are used for interpreting the prediction performance of patient-specific situation classifications. We achieved 90-95% Hamming score values across 24 classifier combinations for 85 different patients used in our experiment. The results are compared with single-label classifiers and without considering the correlations among the vitals. The comparisons show that multi-label method is the best technique for this problem domain. The evaluation results reveal that multi-label classification techniques using the correlations among multiple vitals are effective ways for early estimation of future values of those vitals. In context-aware remote monitoring this process can greatly help the doctors in quick diagnostic decision making. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

A Secure Alignment Algorithm for Mapping Short Reads to Human Genome.

PubMed

Zhao, Yongan; Wang, Xiaofeng; Tang, Haixu

2018-05-09

The elastic and inexpensive computing resources such as clouds have been recognized as a useful solution to analyzing massive human genomic data (e.g., acquired by using next-generation sequencers) in biomedical researches. However, outsourcing human genome computation to public or commercial clouds was hindered due to privacy concerns: even a small number of human genome sequences contain sufficient information for identifying the donor of the genomic data. This issue cannot be directly addressed by existing security and cryptographic techniques (such as homomorphic encryption), because they are too heavyweight to carry out practical genome computation tasks on massive data. In this article, we present a secure algorithm to accomplish the read mapping, one of the most basic tasks in human genomic data analysis based on a hybrid cloud computing model. Comparing with the existing approaches, our algorithm delegates most computation to the public cloud, while only performing encryption and decryption on the private cloud, and thus makes the maximum use of the computing resource of the public cloud. Furthermore, our algorithm reports similar results as the nonsecure read mapping algorithms, including the alignment between reads and the reference genome, which can be directly used in the downstream analysis such as the inference of genomic variations. We implemented the algorithm in C++ and Python on a hybrid cloud system, in which the public cloud uses an Apache Spark system.

An Improved Algorithm for Retrieving Surface Downwelling Longwave Radiation from Satellite Measurements

NASA Technical Reports Server (NTRS)

Zhou, Yaping; Kratz, David P.; Wilber, Anne C.; Gupta, Shashi K.; Cess, Robert D.

2007-01-01

Zhou and Cess [2001] developed an algorithm for retrieving surface downwelling longwave radiation (SDLW) based upon detailed studies using radiative transfer model calculations and surface radiometric measurements. Their algorithm linked clear sky SDLW with surface upwelling longwave flux and column precipitable water vapor. For cloudy sky cases, they used cloud liquid water path as an additional parameter to account for the effects of clouds. Despite the simplicity of their algorithm, it performed very well for most geographical regions except for those regions where the atmospheric conditions near the surface tend to be extremely cold and dry. Systematic errors were also found for scenes that were covered with ice clouds. An improved version of the algorithm prevents the large errors in the SDLW at low water vapor amounts by taking into account that under such conditions the SDLW and water vapor amount are nearly linear in their relationship. The new algorithm also utilizes cloud fraction and cloud liquid and ice water paths available from the Cloud and the Earth's Radiant Energy System (CERES) single scanner footprint (SSF) product to separately compute the clear and cloudy portions of the fluxes. The new algorithm has been validated against surface measurements at 29 stations around the globe for Terra and Aqua satellites. The results show significant improvement over the original version. The revised Zhou-Cess algorithm is also slightly better or comparable to more sophisticated algorithms currently implemented in the CERES processing and will be incorporated as one of the CERES empirical surface radiation algorithms.

Estimating Cloud Cover

ERIC Educational Resources Information Center

Moseley, Christine

2007-01-01

The purpose of this activity was to help students understand the percentage of cloud cover and make more accurate cloud cover observations. Students estimated the percentage of cloud cover represented by simulated clouds and assigned a cloud cover classification to those simulations. (Contains 2 notes and 3 tables.)

A cloud detection algorithm using the downwelling infrared radiance measured by an infrared pyrometer of the ground-based microwave radiometer

DOE PAGES

Ahn, M. H.; Han, D.; Won, H. Y.; ...

2015-02-03

For better utilization of the ground-based microwave radiometer, it is important to detect the cloud presence in the measured data. Here, we introduce a simple and fast cloud detection algorithm by using the optical characteristics of the clouds in the infrared atmospheric window region. The new algorithm utilizes the brightness temperature (Tb) measured by an infrared radiometer installed on top of a microwave radiometer. The two-step algorithm consists of a spectral test followed by a temporal test. The measured Tb is first compared with a predicted clear-sky Tb obtained by an empirical formula as a function of surface air temperaturemore » and water vapor pressure. For the temporal test, the temporal variability of the measured Tb during one minute compares with a dynamic threshold value, representing the variability of clear-sky conditions. It is designated as cloud-free data only when both the spectral and temporal tests confirm cloud-free data. Overall, most of the thick and uniform clouds are successfully detected by the spectral test, while the broken and fast-varying clouds are detected by the temporal test. The algorithm is validated by comparison with the collocated ceilometer data for six months, from January to June 2013. The overall proportion of correctness is about 88.3% and the probability of detection is 90.8%, which are comparable with or better than those of previous similar approaches. Two thirds of discrepancies occur when the new algorithm detects clouds while the ceilometer does not, resulting in different values of the probability of detection with different cloud-base altitude, 93.8, 90.3, and 82.8% for low, mid, and high clouds, respectively. Finally, due to the characteristics of the spectral range, the new algorithm is found to be insensitive to the presence of inversion layers.« less

Street curb recognition in 3d point cloud data using morphological operations

NASA Astrophysics Data System (ADS)

Rodríguez-Cuenca, Borja; Concepción Alonso-Rodríguez, María; García-Cortés, Silverio; Ordóñez, Celestino

2015-04-01

Accurate and automatic detection of cartographic-entities saves a great deal of time and money when creating and updating cartographic databases. The current trend in remote sensing feature extraction is to develop methods that are as automatic as possible. The aim is to develop algorithms that can obtain accurate results with the least possible human intervention in the process. Non-manual curb detection is an important issue in road maintenance, 3D urban modeling, and autonomous navigation fields. This paper is focused on the semi-automatic recognition of curbs and street boundaries using a 3D point cloud registered by a mobile laser scanner (MLS) system. This work is divided into four steps. First, a coordinate system transformation is carried out, moving from a global coordinate system to a local one. After that and in order to simplify the calculations involved in the procedure, a rasterization based on the projection of the measured point cloud on the XY plane was carried out, passing from the 3D original data to a 2D image. To determine the location of curbs in the image, different image processing techniques such as thresholding and morphological operations were applied. Finally, the upper and lower edges of curbs are detected by an unsupervised classification algorithm on the curvature and roughness of the points that represent curbs. The proposed method is valid in both straight and curved road sections and applicable both to laser scanner and stereo vision 3D data due to the independence of its scanning geometry. This method has been successfully tested with two datasets measured by different sensors. The first dataset corresponds to a point cloud measured by a TOPCON sensor in the Spanish town of Cudillero. That point cloud comprises more than 6,000,000 points and covers a 400-meter street. The second dataset corresponds to a point cloud measured by a RIEGL sensor in the Austrian town of Horn. That point cloud comprises 8,000,000 points and represents a 160-meter street. The proposed method provides success rates in curb recognition of over 85% in both datasets.

Cloud cover determination in polar regions from satellite imagery

NASA Technical Reports Server (NTRS)

Barry, R. G.; Maslanik, J. A.; Key, J. R.

1987-01-01

A definition is undertaken of the spectral and spatial characteristics of clouds and surface conditions in the polar regions, and to the creation of calibrated, geometrically correct data sets suitable for quantitative analysis. Ways are explored in which this information can be applied to cloud classifications as new methods or as extensions to existing classification schemes. A methodology is developed that uses automated techniques to merge Advanced Very High Resolution Radiometer (AVHRR) and Scanning Multichannel Microwave Radiometer (SMMR) data, and to apply first-order calibration and zenith angle corrections to the AVHRR imagery. Cloud cover and surface types are manually interpreted, and manual methods are used to define relatively pure training areas to describe the textural and multispectral characteristics of clouds over several surface conditions. The effects of viewing angle and bidirectional reflectance differences are studied for several classes, and the effectiveness of some key components of existing classification schemes is tested.

Reflections on current and future applications of multiangle imaging to aerosol and cloud remote sensing

NASA Astrophysics Data System (ADS)

Diner, David

2010-05-01

The Multi-angle Imaging SpectroRadiometer (MISR) instrument has been collecting global Earth data from NASA's Terra satellite since February 2000. With its 9 along-track view angles, 4 spectral bands, intrinsic spatial resolution of 275 m, and stable radiometric and geometric calibration, no instrument that combines MISR's attributes has previously flown in space, nor is there is a similar capability currently available on any other satellite platform. Multiangle imaging offers several tools for remote sensing of aerosol and cloud properties, including bidirectional reflectance and scattering measurements, stereoscopic pattern matching, time lapse sequencing, and potentially, optical tomography. Current data products from MISR employ several of these techniques. Observations of the intensity of scattered light as a function of view angle and wavelength provide accurate measures of aerosol optical depths (AOD) over land, including bright desert and urban source regions. Partitioning of AOD according to retrieved particle classification and incorporation of height information improves the relationship between AOD and surface PM2.5 (fine particulate matter, a regulated air pollutant), constituting an important step toward a satellite-based particulate pollution monitoring system. Stereoscopic cloud-top heights provide a unique metric for detecting interannual variability of clouds and exceptionally high quality and sensitivity for detection and height retrieval for low-level clouds. Using the several-minute time interval between camera views, MISR has enabled a pole-to-pole, height-resolved atmospheric wind measurement system. Stereo imagery also makes possible global measurement of the injection heights and advection speeds of smoke plumes, volcanic plumes, and dust clouds, for which a large database is now available. To build upon what has been learned during the first decade of MISR observations, we are evaluating algorithm updates that not only refine retrieval accuracies but also include enhancements (e.g., finer spatial resolution) that would have been computationally prohibitive just ten years ago. In addition, we are developing technological building blocks for future sensors that enable broader spectral coverage, wider swath, and incorporation of high-accuracy polarimetric imaging. Prototype cameras incorporating photoelastic modulators have been constructed. To fully capitalize on the rich information content of the current and next-generation of multiangle imagers, several algorithmic paradigms currently employed need to be re-examined, e.g., the use of aerosol look-up tables, neglect of 3-D effects, and binary partitioning of the atmosphere into "cloudy" or "clear" designations. Examples of progress in algorithm and technology developments geared toward advanced application of multiangle imaging to remote sensing of aerosols and clouds will be presented.

Retrieval of radiative and microphysical properties of clouds from multispectral infrared measurements

NASA Astrophysics Data System (ADS)

Iwabuchi, Hironobu; Saito, Masanori; Tokoro, Yuka; Putri, Nurfiena Sagita; Sekiguchi, Miho

2016-12-01

Satellite remote sensing of the macroscopic, microphysical, and optical properties of clouds are useful for studying spatial and temporal variations of clouds at various scales and constraining cloud physical processes in climate and weather prediction models. Instead of using separate independent algorithms for different cloud properties, a unified, optimal estimation-based cloud retrieval algorithm is developed and applied to moderate resolution imaging spectroradiometer (MODIS) observations using ten thermal infrared bands. The model considers sensor configurations, background surface and atmospheric profile, and microphysical and optical models of ice and liquid cloud particles and radiative transfer in a plane-parallel, multilayered atmosphere. Measurement and model errors are thoroughly quantified from direct comparisons of clear-sky observations over the ocean with model calculations. Performance tests by retrieval simulations show that ice cloud properties are retrieved with high accuracy when cloud optical thickness (COT) is between 0.1 and 10. Cloud-top pressure is inferred with uncertainty lower than 10 % when COT is larger than 0.3. Applying the method to a tropical cloud system and comparing the results with the MODIS Collection 6 cloud product shows good agreement for ice cloud optical thickness when COT is less than about 5. Cloud-top height agrees well with estimates obtained by the CO2 slicing method used in the MODIS product. The present algorithm can detect optically thin parts at the edges of high clouds well in comparison with the MODIS product, in which these parts are recognized as low clouds by the infrared window method. The cloud thermodynamic phase in the present algorithm is constrained by cloud-top temperature, which tends not to produce results with an ice cloud that is too warm and liquid cloud that is too cold.

Investigation of hydrometeor classification uncertainties through the POLARRIS polarimetric radar simulator

NASA Astrophysics Data System (ADS)

Dolan, B.; Rutledge, S. A.; Barnum, J. I.; Matsui, T.; Tao, W. K.; Iguchi, T.

2017-12-01

POLarimetric Radar Retrieval and Instrument Simulator (POLARRIS) is a framework that has been developed to simulate radar observations from cloud resolving model (CRM) output and subject model data and observations to the same retrievals, analysis and visualization. This framework not only enables validation of bulk microphysical model simulated properties, but also offers an opportunity to study the uncertainties associated with retrievals such as hydrometeor classification (HID). For the CSU HID, membership beta functions (MBFs) are built using a set of simulations with realistic microphysical assumptions about axis ratio, density, canting angles, size distributions for each of ten hydrometeor species. These assumptions are tested using POLARRIS to understand their influence on the resulting simulated polarimetric data and final HID classification. Several of these parameters (density, size distributions) are set by the model microphysics, and therefore the specific assumptions of axis ratio and canting angle are carefully studied. Through these sensitivity studies, we hope to be able to provide uncertainties in retrieved polarimetric variables and HID as applied to CRM output. HID retrievals assign a classification to each point by determining the highest score, thereby identifying the dominant hydrometeor type within a volume. However, in nature, there is rarely just one a single hydrometeor type at a particular point. Models allow for mixing ratios of different hydrometeors within a grid point. We use the mixing ratios from CRM output in concert with the HID scores and classifications to understand how the HID algorithm can provide information about mixtures within a volume, as well as calculate a confidence in the classifications. We leverage the POLARRIS framework to additionally probe radar wavelength differences toward the possibility of a multi-wavelength HID which could utilize the strengths of different wavelengths to improve HID classifications. With these uncertainties and algorithm improvements, cases of convection are studied in a continental (Oklahoma) and maritime (Darwin, Australia) regime. Observations from C-band polarimetric data in both locations are compared to CRM simulations from NU-WRF using the POLARRIS framework.

APOLLO_NG - a probabilistic interpretation of the APOLLO legacy for AVHRR heritage channels

NASA Astrophysics Data System (ADS)

Klüser, L.; Killius, N.; Gesell, G.

2015-04-01

The cloud processing scheme APOLLO (Avhrr Processing scheme Over cLouds, Land and Ocean) has been in use for cloud detection and cloud property retrieval since the late 1980s. The physics of the APOLLO scheme still build the backbone of a range of cloud detection algorithms for AVHRR (Advanced Very High Resolution Radiometer) heritage instruments. The APOLLO_NG (APOLLO_NextGeneration) cloud processing scheme is a probabilistic interpretation of the original APOLLO method. While building upon the physical principles having served well in the original APOLLO a couple of additional variables have been introduced in APOLLO_NG. Cloud detection is not performed as a binary yes/no decision based on these physical principals but is expressed as cloud probability for each satellite pixel. Consequently the outcome of the algorithm can be tuned from clear confident to cloud confident depending on the purpose. The probabilistic approach allows to retrieving not only the cloud properties (optical depth, effective radius, cloud top temperature and cloud water path) but also their uncertainties. APOLLO_NG is designed as a standalone cloud retrieval method robust enough for operational near-realtime use and for the application with large amounts of historical satellite data. Thus the radiative transfer solution is approximated by the same two stream approach which also had been used for the original APOLLO. This allows the algorithm to be robust enough for being applied to a wide range of sensors without the necessity of sensor-specific tuning. Moreover it allows for online calculation of the radiative transfer (i.e. within the retrieval algorithm) giving rise to a detailed probabilistic treatment of cloud variables. This study presents the algorithm for cloud detection and cloud property retrieval together with the physical principles from the APOLLO legacy it is based on. Furthermore a couple of example results from on NOAA-18 are presented.

Evaluation of the OMI Cloud Pressures Derived from Rotational Raman Scattering by Comparisons with other Satellite Data and Radiative Transfer Simulations

NASA Technical Reports Server (NTRS)

Vasilkov, Alexander; Joiner, Joanna; Spurr, Robert; Bhartia, Pawan K.; Levelt, Pieternel; Stephens, Graeme

2009-01-01

In this paper we examine differences between cloud pressures retrieved from the Ozone Monitoring Instrument (OMI) using the ultraviolet rotational Raman scattering (RRS) algorithm and those from the thermal infrared (IR) Aqua/MODIS. Several cloud data sets are currently being used in OMI trace gas retrieval algorithms including climatologies based on IR measurements and simultaneous cloud parameters derived from OMI. From a validation perspective, it is important to understand the OMI retrieved cloud parameters and how they differ with those derived from the IR. To this end, we perform radiative transfer calculations to simulate the effects of different geophysical conditions on the OMI RRS cloud pressure retrievals. We also quantify errors related to the use of the Mixed Lambert-Equivalent Reflectivity (MLER) concept as currently implemented of the OMI algorithms. Using properties from the Cloudsat radar and MODIS, we show that radiative transfer calculations support the following: (1) The MLER model is adequate for single-layer optically thick, geometrically thin clouds, but can produce significant errors in estimated cloud pressure for optically thin clouds. (2) In a two-layer cloud, the RRS algorithm may retrieve a cloud pressure that is either between the two cloud decks or even beneath the top of the lower cloud deck because of scattering between the cloud layers; the retrieved pressure depends upon the viewing geometry and the optical depth of the upper cloud deck. (3) Absorbing aerosol in and above a cloud can produce significant errors in the retrieved cloud pressure. (4) The retrieved RRS effective pressure for a deep convective cloud will be significantly higher than the physical cloud top pressure derived with thermal IR.

Progress towards MODIS and VIIRS Cloud Optical Property Data Record Continuity

NASA Astrophysics Data System (ADS)

Meyer, K.; Platnick, S. E.; Wind, G.; Amarasinghe, N.; Holz, R.; Ackerman, S. A.; Heidinger, A. K.

2016-12-01

The launch of Suomi NPP in the fall of 2011 began the next generation of U.S. operational polar orbiting Earth observations, and its VIIRS imager provides an opportunity to extend the 15+ year climate data record of MODIS EOS. Similar to MODIS, VIIRS provides visible through IR observations at moderate spatial resolution with a 1330 LT equatorial crossing consistent with the MODIS on the Aqua platform. However, unlike MODIS, VIIRS lacks key water vapor and CO2 absorbing channels used for high cloud detection and cloud-top property retrievals, and there is a significant change in the spectral location of the 2.1μm shortwave-infrared channel used for cloud optical/microphysical retrievals and cloud thermodynamic phase. Given these instrument differences between MODIS EOS and VIIRS S-NPP/JPSS, we discuss our progress towards merging the MODIS observational record with VIIRS in order to generate cloud optical property climate data record continuity across the observing systems. The MODIS-VIIRS continuity algorithm for cloud optical property retrievals leverages heritage algorithms that produce the existing MODIS cloud optical and microphysical properties product (MOD06); the NOAA AWG/CLAVR-x cloud-top property algorithm and a common MODIS-VIIRS cloud mask feed into the optical property algorithm. To account for the different channel sets of MODIS and VIIRS, each algorithm nominally uses a subset of channels common to both imagers. Data granule and aggregated examples for the current version of the continuity algorithm (MODAWG) will be shown. In addition, efforts to reconcile apparent radiometric biases between analogous channels of the two imagers, a critical consideration for obtaining inter-sensor climate data record continuity, will be discussed.

Orbiting Carbon Observatory-2 (OCO-2) cloud screening algorithms: validation against collocated MODIS and CALIOP data

NASA Astrophysics Data System (ADS)

Taylor, Thomas E.; O'Dell, Christopher W.; Frankenberg, Christian; Partain, Philip T.; Cronk, Heather Q.; Savtchenko, Andrey; Nelson, Robert R.; Rosenthal, Emily J.; Chang, Albert Y.; Fisher, Brenden; Osterman, Gregory B.; Pollock, Randy H.; Crisp, David; Eldering, Annmarie; Gunson, Michael R.

2016-03-01

The objective of the National Aeronautics and Space Administration's (NASA) Orbiting Carbon Observatory-2 (OCO-2) mission is to retrieve the column-averaged carbon dioxide (CO2) dry air mole fraction (XCO2) from satellite measurements of reflected sunlight in the near-infrared. These estimates can be biased by clouds and aerosols, i.e., contamination, within the instrument's field of view. Screening of the most contaminated soundings minimizes unnecessary calls to the computationally expensive Level 2 (L2) XCO2 retrieval algorithm. Hence, robust cloud screening methods have been an important focus of the OCO-2 algorithm development team. Two distinct, computationally inexpensive cloud screening algorithms have been developed for this application. The A-Band Preprocessor (ABP) retrieves the surface pressure using measurements in the 0.76 µm O2 A band, neglecting scattering by clouds and aerosols, which introduce photon path-length differences that can cause large deviations between the expected and retrieved surface pressure. The Iterative Maximum A Posteriori (IMAP) Differential Optical Absorption Spectroscopy (DOAS) Preprocessor (IDP) retrieves independent estimates of the CO2 and H2O column abundances using observations taken at 1.61 µm (weak CO2 band) and 2.06 µm (strong CO2 band), while neglecting atmospheric scattering. The CO2 and H2O column abundances retrieved in these two spectral regions differ significantly in the presence of cloud and scattering aerosols. The combination of these two algorithms, which are sensitive to different features in the spectra, provides the basis for cloud screening of the OCO-2 data set.To validate the OCO-2 cloud screening approach, collocated measurements from NASA's Moderate Resolution Imaging Spectrometer (MODIS), aboard the Aqua platform, were compared to results from the two OCO-2 cloud screening algorithms. With tuning of algorithmic threshold parameters that allows for processing of ≃ 20-25 % of all OCO-2 soundings, agreement between the OCO-2 and MODIS cloud screening methods is found to be ≃ 85 % over four 16-day orbit repeat cycles in both the winter (December) and spring (April-May) for OCO-2 nadir-land, glint-land and glint-water observations.No major, systematic, spatial or temporal dependencies were found, although slight differences in the seasonal data sets do exist and validation is more problematic with increasing solar zenith angle and when surfaces are covered in snow and ice and have complex topography. To further analyze the performance of the cloud screening algorithms, an initial comparison of OCO-2 observations was made to collocated measurements from the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) aboard the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO). These comparisons highlight the strength of the OCO-2 cloud screening algorithms in identifying high, thin clouds but suggest some difficulty in identifying some clouds near the surface, even when the optical thicknesses are greater than 1.

Evaluation of automatic cloud removal method for high elevation areas in Landsat 8 OLI images to improve environmental indexes computation

NASA Astrophysics Data System (ADS)

Alvarez, César I.; Teodoro, Ana; Tierra, Alfonso

2017-10-01

Thin clouds in the optical remote sensing data are frequent and in most of the cases don't allow to have a pure surface data in order to calculate some indexes as Normalized Difference Vegetation Index (NDVI). This paper aims to evaluate the Automatic Cloud Removal Method (ACRM) algorithm over a high elevation city like Quito (Ecuador), with an altitude of 2800 meters above sea level, where the clouds are presented all the year. The ACRM is an algorithm that considers a linear regression between each Landsat 8 OLI band and the Cirrus band using the slope obtained with the linear regression established. This algorithm was employed without any reference image or mask to try to remove the clouds. The results of the application of the ACRM algorithm over Quito didn't show a good performance. Therefore, was considered improving this algorithm using a different slope value data (ACMR Improved). After, the NDVI computation was compared with a reference NDVI MODIS data (MOD13Q1). The ACMR Improved algorithm had a successful result when compared with the original ACRM algorithm. In the future, this Improved ACRM algorithm needs to be tested in different regions of the world with different conditions to evaluate if the algorithm works successfully for all conditions.

The Day-1 GPM Combined Precipitation Algorithm: IMERG

NASA Astrophysics Data System (ADS)

Huffman, G. J.; Bolvin, D. T.; Braithwaite, D.; Hsu, K.; Joyce, R.; Kidd, C.; Sorooshian, S.; Xie, P.

2012-12-01

The Integrated Multi-satellitE Retrievals for Global Precipitation Measurement (GPM) mission (IMERG) algorithm will provide the at-launch combined-sensor precipitation dataset being produced by the U.S. GPM Science Team. IMERG is being developed as a unified U.S. algorithm that takes advantage of strengths in three current U.S. algorithms: - the TRMM Multi-satellite Precipitation Analysis (TMPA), which addresses inter-satellite calibration of precipitation estimates and monthly scale combination of satellite and gauge analyses; - the CPC Morphing algorithm with Kalman Filtering (KF-CMORPH), which provides quality-weighted time interpolation of precipitation patterns following storm motion; and - the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks using a Cloud Classification System (PERSIANN-CCS), which provides a neural-network-based scheme for generating microwave-calibrated precipitation estimates from geosynchronous infrared brightness temperatures, and filters out some non-raining cold clouds. The goal is to provide a long-term, fine-scale record of global precipitation from the entire constellation of precipitation-relevant satellite sensors, with input from surface precipitation gauges. The record will begin January 1998 at the start of the Tropical Rainfall Measuring Mission (TRMM) and extend as GPM records additional data. Although homogeneity is considered desirable, the use of diverse and evolving data sources works against the strict long-term homogeneity that characterizes a Climate Data Record (CDR). This talk will briefly review the design requirements for IMERG, including multiple runs at different latencies (most likely around 4 hours, 12 hours, and 2 months after observation time), various intermediate data fields as part of the IMERG data file, and the plans to bring up IMERG with calibration by TRMM initially, transitioning to GPM when its individual-sensor precipitation algorithms are fully functional. Then we will present some early examples of IMERG data products and compare them with existing products to illustrate how the design of IMERG affects the overall performance of the algorithm.

Cloud cover estimation optical package: New facility, algorithms and techniques

NASA Astrophysics Data System (ADS)

Krinitskiy, Mikhail

2017-02-01

Short- and long-wave radiation is an important component of surface heat budget over sea and land. For estimating them accurate observations of the cloud cover are needed. While massively observed visually, for building accurate parameterizations cloud cover needs also to be quantified using precise instrumental measurements. Major disadvantages of the most of existing cloud-cameras are associated with their complicated design and inaccuracy of post-processing algorithms which typically result in the uncertainties of 20% to 30% in the camera-based estimates of cloud cover. The accuracy of these types of algorithm in terms of true scoring compared to human-observed values is typically less than 10%. We developed new generation package for cloud cover estimating, which provides much more accurate results and also allows for measuring additional characteristics. New algorithm, namely SAIL GrIx, based on routine approach, also developed for this package. It uses the synthetic controlling index ("grayness rate index") which allows to suppress the background sunburn effect. This makes it possible to increase the reliability of the detection of the optically thin clouds. The accuracy of this algorithm in terms of true scoring became 30%. One more approach, namely SAIL GrIx ML, we have used to increase the cloud cover estimating accuracy is the algorithm that uses machine learning technique along with some other signal processing techniques. Sun disk condition appears to be a strong feature in this kind of models. Artificial Neural Networks type of model demonstrates the best quality. This model accuracy in terms of true scoring increases up to 95,5%. Application of a new algorithm lets us to modify the design of the optical sensing package and to avoid the use of the solar trackers. This made the design of the cloud camera much more compact. New cloud-camera has already been tested in several missions across Atlantic and Indian oceans on board of IORAS research vessels.

An improved algorithm for wildfire detection

NASA Astrophysics Data System (ADS)

Nakau, K.

2010-12-01

Satellite information of wild fire location has strong demands from society. Therefore, Understanding such demands is quite important to consider what to improve the wild fire detection algorithm. Interviews and considerations imply that the most important improvements are geographical resolution of the wildfire product and classification of fire; smoldering or flaming. Discussion with fire service agencies are performed with fire service agencies in Alaska and fire service volunteer groups in Indonesia. Alaska Fire Service (AFS) makes 3D-map overlaid by fire location every morning. Then, this 3D-map is examined by leaders of fire service teams to decide their strategy to fighting against wild fire. Especially, firefighters of both agencies seek the best walk path to approach the fire. Because of mountainous landscape, geospatial resolution is quite important for them. For example, walking in bush for 1km, as same as one pixel of fire product, is very tough for firefighters. Also, in case of remote wild fire, fire service agencies utilize satellite information to decide when to have a flight observation to confirm the status; expanding, flaming, smoldering or out. Therefore, it is also quite important to provide the classification of fire; flaming or smoldering. Not only the aspect of disaster management, wildfire emits huge amount of carbon into atmosphere as much as one quarter to one half of CO2 by fuel combustion (IPCC AR4). Reduction of the CO2 emission by human caused wildfire is important. To estimate carbon emission from wildfire, special resolution is quite important. To improve sensitivity of wild fire detection, author adopts radiance based wildfire detection. Different from the existing brightness temperature approach, we can easily consider reflectance of background land coverage. Especially for GCOM-C1/SGLI, band to detect fire with 250m resolution is 1.6μm wavelength. In this band, we have much more sunlight reflection. Therefore, we need to consider the way to cancel sunlight reflection. In this study, author utilizes simple linear correction for estimation of infrared emission considering sunlight reflection. As well as bran new core part of wildfire algorithm, we need to eliminate bright reflectance matters, including cloud, desert and sun glint. Also, we need to eliminate the false alarms at coastal area for difference of surface temperature between land and ocean. An existing algorithm MOD14 has same procedure, however, some of these ancillary parts are newly introduced or improved. Snow mask is newly introduced to reduce a bright reflectance with snow and ice covered area. Also, the improved ancillary parts include candidate selection of fire pixel, cloud mask, water body mask. With these improvements, wildfire with dense smoke or wildfire under thin cloud could be detected by this algorithm. This wild fire product is not validated by ground observations yet. However, distribution is well corresponded with wildfire location in same periods. Unfortunately, this algorithm also detects false alarm in urban area same as existing one. This should be corrected adopting other bands. Current algorithm will be performed in JASMES website.

APOLLO_NG - a probabilistic interpretation of the APOLLO legacy for AVHRR heritage channels

NASA Astrophysics Data System (ADS)

Klüser, L.; Killius, N.; Gesell, G.

2015-10-01

The cloud processing scheme APOLLO (AVHRR Processing scheme Over cLouds, Land and Ocean) has been in use for cloud detection and cloud property retrieval since the late 1980s. The physics of the APOLLO scheme still build the backbone of a range of cloud detection algorithms for AVHRR (Advanced Very High Resolution Radiometer) heritage instruments. The APOLLO_NG (APOLLO_NextGeneration) cloud processing scheme is a probabilistic interpretation of the original APOLLO method. It builds upon the physical principles that have served well in the original APOLLO scheme. Nevertheless, a couple of additional variables have been introduced in APOLLO_NG. Cloud detection is no longer performed as a binary yes/no decision based on these physical principles. It is rather expressed as cloud probability for each satellite pixel. Consequently, the outcome of the algorithm can be tuned from being sure to reliably identify clear pixels to conditions of reliably identifying definitely cloudy pixels, depending on the purpose. The probabilistic approach allows retrieving not only the cloud properties (optical depth, effective radius, cloud top temperature and cloud water path) but also their uncertainties. APOLLO_NG is designed as a standalone cloud retrieval method robust enough for operational near-realtime use and for application to large amounts of historical satellite data. The radiative transfer solution is approximated by the same two-stream approach which also had been used for the original APOLLO. This allows the algorithm to be applied to a wide range of sensors without the necessity of sensor-specific tuning. Moreover it allows for online calculation of the radiative transfer (i.e., within the retrieval algorithm) giving rise to a detailed probabilistic treatment of cloud variables. This study presents the algorithm for cloud detection and cloud property retrieval together with the physical principles from the APOLLO legacy it is based on. Furthermore a couple of example results from NOAA-18 are presented.

Detection of supercooled liquid water-topped mixed-phase clouds >from shortwave-infrared satellite observations

NASA Astrophysics Data System (ADS)

NOH, Y. J.; Miller, S. D.; Heidinger, A. K.

2015-12-01

Many studies have demonstrated the utility of multispectral information from satellite passive radiometers for detecting and retrieving the properties of cloud globally, which conventionally utilizes shortwave- and thermal-infrared bands. However, the satellite-derived cloud information comes mainly from cloud top or represents a vertically integrated property. This can produce a large bias in determining cloud phase characteristics, in particular for mixed-phase clouds which are often observed to have supercooled liquid water at cloud top but a predominantly ice phase residing below. The current satellite retrieval algorithms may report these clouds simply as supercooled liquid without any further information regarding the presence of a sub-cloud-top ice phase. More accurate characterization of these clouds is very important for climate models and aviation applications. In this study, we present a physical basis and preliminary results for the algorithm development of supercooled liquid-topped mixed-phase cloud detection using satellite radiometer observations. The detection algorithm is based on differential absorption properties between liquid and ice particles in the shortwave-infrared bands. Solar reflectance data in narrow bands at 1.6 μm and 2.25 μm are used to optically probe below clouds for distinction between supercooled liquid-topped clouds with and without an underlying mixed phase component. Varying solar/sensor geometry and cloud optical properties are also considered. The spectral band combination utilized for the algorithm is currently available on Suomi NPP Visible/Infrared Imaging Radiometer Suite (VIIRS), Himawari-8 Advanced Himawari Imager (AHI), and the future GOES-R Advance Baseline Imager (ABI). When tested on simulated cloud fields from WRF model and synthetic ABI data, favorable results were shown with reasonable threat scores (0.6-0.8) and false alarm rates (0.1-0.2). An ARM/NSA case study applied to VIIRS data also indicated promising potential of the algorithm.

Multi-Spectral Cloud Retrievals from Moderate Image Spectrometer (MODIS)

NASA Technical Reports Server (NTRS)

Platnick, Steven

2004-01-01

MODIS observations from the NASA EOS Terra spacecraft (1030 local time equatorial sun-synchronous crossing) launched in December 1999 have provided a unique set of Earth observation data. With the launch of the NASA EOS Aqua spacecraft (1330 local time crossing! in May 2002: two MODIS daytime (sunlit) and nighttime observations are now available in a 24-hour period allowing some measure of diurnal variability. A comprehensive set of remote sensing algorithms for cloud masking and the retrieval of cloud physical and optical properties has been developed by members of the MODIS atmosphere science team. The archived products from these algorithms have applications in climate modeling, climate change studies, numerical weather prediction, as well as fundamental atmospheric research. In addition to an extensive cloud mask, products include cloud-top properties (temperature, pressure, effective emissivity), cloud thermodynamic phase, cloud optical and microphysical parameters (optical thickness, effective particle radius, water path), as well as derived statistics. An overview of the instrument and cloud algorithms will be presented along with various examples, including an initial analysis of several operational global gridded (Level-3) cloud products from the two platforms. Statistics of cloud optical and microphysical properties as a function of latitude for land and Ocean regions will be shown. Current algorithm research efforts will also be discussed.

An Effective Mechanism for Virtual Machine Placement using Aco in IAAS Cloud

NASA Astrophysics Data System (ADS)

Shenbaga Moorthy, Rajalakshmi; Fareentaj, U.; Divya, T. K.

2017-08-01

Cloud computing provides an effective way to dynamically provide numerous resources to meet customer demands. A major challenging problem for cloud providers is designing efficient mechanisms for optimal virtual machine Placement (OVMP). Such mechanisms enable the cloud providers to effectively utilize their available resources and obtain higher profits. In order to provide appropriate resources to the clients an optimal virtual machine placement algorithm is proposed. Virtual machine placement is NP-Hard problem. Such NP-Hard problem can be solved using heuristic algorithm. In this paper, Ant Colony Optimization based virtual machine placement is proposed. Our proposed system focuses on minimizing the cost spending in each plan for hosting virtual machines in a multiple cloud provider environment and the response time of each cloud provider is monitored periodically, in such a way to minimize delay in providing the resources to the users. The performance of the proposed algorithm is compared with greedy mechanism. The proposed algorithm is simulated in Eclipse IDE. The results clearly show that the proposed algorithm minimizes the cost, response time and also number of migrations.

Application-oriented offloading in heterogeneous networks for mobile cloud computing

NASA Astrophysics Data System (ADS)

Tseng, Fan-Hsun; Cho, Hsin-Hung; Chang, Kai-Di; Li, Jheng-Cong; Shih, Timothy K.

2018-04-01

Nowadays Internet applications have become more complicated that mobile device needs more computing resources for shorter execution time but it is restricted to limited battery capacity. Mobile cloud computing (MCC) is emerged to tackle the finite resource problem of mobile device. MCC offloads the tasks and jobs of mobile devices to cloud and fog environments by using offloading scheme. It is vital to MCC that which task should be offloaded and how to offload efficiently. In the paper, we formulate the offloading problem between mobile device and cloud data center and propose two algorithms based on application-oriented for minimum execution time, i.e. the Minimum Offloading Time for Mobile device (MOTM) algorithm and the Minimum Execution Time for Cloud data center (METC) algorithm. The MOTM algorithm minimizes offloading time by selecting appropriate offloading links based on application categories. The METC algorithm minimizes execution time in cloud data center by selecting virtual and physical machines with corresponding resource requirements of applications. Simulation results show that the proposed mechanism not only minimizes total execution time for mobile devices but also decreases their energy consumption.

Automatic Matching of Large Scale Images and Terrestrial LIDAR Based on App Synergy of Mobile Phone

NASA Astrophysics Data System (ADS)

Xia, G.; Hu, C.

2018-04-01

The digitalization of Cultural Heritage based on ground laser scanning technology has been widely applied. High-precision scanning and high-resolution photography of cultural relics are the main methods of data acquisition. The reconstruction with the complete point cloud and high-resolution image requires the matching of image and point cloud, the acquisition of the homonym feature points, the data registration, etc. However, the one-to-one correspondence between image and corresponding point cloud depends on inefficient manual search. The effective classify and management of a large number of image and the matching of large image and corresponding point cloud will be the focus of the research. In this paper, we propose automatic matching of large scale images and terrestrial LiDAR based on APP synergy of mobile phone. Firstly, we develop an APP based on Android, take pictures and record related information of classification. Secondly, all the images are automatically grouped with the recorded information. Thirdly, the matching algorithm is used to match the global and local image. According to the one-to-one correspondence between the global image and the point cloud reflection intensity image, the automatic matching of the image and its corresponding laser radar point cloud is realized. Finally, the mapping relationship between global image, local image and intensity image is established according to homonym feature point. So we can establish the data structure of the global image, the local image in the global image, the local image corresponding point cloud, and carry on the visualization management and query of image.

Towards Automated Large-Scale 3D Phenotyping of Vineyards under Field Conditions

PubMed Central

Rose, Johann Christian; Kicherer, Anna; Wieland, Markus; Klingbeil, Lasse; Töpfer, Reinhard; Kuhlmann, Heiner

2016-01-01

In viticulture, phenotypic data are traditionally collected directly in the field via visual and manual means by an experienced person. This approach is time consuming, subjective and prone to human errors. In recent years, research therefore has focused strongly on developing automated and non-invasive sensor-based methods to increase data acquisition speed, enhance measurement accuracy and objectivity and to reduce labor costs. While many 2D methods based on image processing have been proposed for field phenotyping, only a few 3D solutions are found in the literature. A track-driven vehicle consisting of a camera system, a real-time-kinematic GPS system for positioning, as well as hardware for vehicle control, image storage and acquisition is used to visually capture a whole vine row canopy with georeferenced RGB images. In the first post-processing step, these images were used within a multi-view-stereo software to reconstruct a textured 3D point cloud of the whole grapevine row. A classification algorithm is then used in the second step to automatically classify the raw point cloud data into the semantic plant components, grape bunches and canopy. In the third step, phenotypic data for the semantic objects is gathered using the classification results obtaining the quantity of grape bunches, berries and the berry diameter. PMID:27983669

Towards Automated Large-Scale 3D Phenotyping of Vineyards under Field Conditions.

PubMed

Rose, Johann Christian; Kicherer, Anna; Wieland, Markus; Klingbeil, Lasse; Töpfer, Reinhard; Kuhlmann, Heiner

2016-12-15

In viticulture, phenotypic data are traditionally collected directly in the field via visual and manual means by an experienced person. This approach is time consuming, subjective and prone to human errors. In recent years, research therefore has focused strongly on developing automated and non-invasive sensor-based methods to increase data acquisition speed, enhance measurement accuracy and objectivity and to reduce labor costs. While many 2D methods based on image processing have been proposed for field phenotyping, only a few 3D solutions are found in the literature. A track-driven vehicle consisting of a camera system, a real-time-kinematic GPS system for positioning, as well as hardware for vehicle control, image storage and acquisition is used to visually capture a whole vine row canopy with georeferenced RGB images. In the first post-processing step, these images were used within a multi-view-stereo software to reconstruct a textured 3D point cloud of the whole grapevine row. A classification algorithm is then used in the second step to automatically classify the raw point cloud data into the semantic plant components, grape bunches and canopy. In the third step, phenotypic data for the semantic objects is gathered using the classification results obtaining the quantity of grape bunches, berries and the berry diameter.

Algorithm for Automated Detection of Edges of Clouds

NASA Technical Reports Server (NTRS)

Ward, Jennifer G.; Merceret, Francis J.

2006-01-01

An algorithm processes cloud-physics data gathered in situ by an aircraft, along with reflectivity data gathered by ground-based radar, to determine whether the aircraft is inside or outside a cloud at a given time. A cloud edge is deemed to be detected when the in/out state changes, subject to a hysteresis constraint. Such determinations are important in continuing research on relationships among lightning, electric charges in clouds, and decay of electric fields with distance from cloud edges.

Contextual Classification of Point Cloud Data by Exploiting Individual 3d Neigbourhoods

NASA Astrophysics Data System (ADS)

Weinmann, M.; Schmidt, A.; Mallet, C.; Hinz, S.; Rottensteiner, F.; Jutzi, B.

2015-03-01

The fully automated analysis of 3D point clouds is of great importance in photogrammetry, remote sensing and computer vision. For reliably extracting objects such as buildings, road inventory or vegetation, many approaches rely on the results of a point cloud classification, where each 3D point is assigned a respective semantic class label. Such an assignment, in turn, typically involves statistical methods for feature extraction and machine learning. Whereas the different components in the processing workflow have extensively, but separately been investigated in recent years, the respective connection by sharing the results of crucial tasks across all components has not yet been addressed. This connection not only encapsulates the interrelated issues of neighborhood selection and feature extraction, but also the issue of how to involve spatial context in the classification step. In this paper, we present a novel and generic approach for 3D scene analysis which relies on (i) individually optimized 3D neighborhoods for (ii) the extraction of distinctive geometric features and (iii) the contextual classification of point cloud data. For a labeled benchmark dataset, we demonstrate the beneficial impact of involving contextual information in the classification process and that using individual 3D neighborhoods of optimal size significantly increases the quality of the results for both pointwise and contextual classification.

Classification of Aerial Photogrammetric 3d Point Clouds

NASA Astrophysics Data System (ADS)

Becker, C.; Häni, N.; Rosinskaya, E.; d'Angelo, E.; Strecha, C.

2017-05-01

We present a powerful method to extract per-point semantic class labels from aerial photogrammetry data. Labelling this kind of data is important for tasks such as environmental modelling, object classification and scene understanding. Unlike previous point cloud classification methods that rely exclusively on geometric features, we show that incorporating color information yields a significant increase in accuracy in detecting semantic classes. We test our classification method on three real-world photogrammetry datasets that were generated with Pix4Dmapper Pro, and with varying point densities. We show that off-the-shelf machine learning techniques coupled with our new features allow us to train highly accurate classifiers that generalize well to unseen data, processing point clouds containing 10 million points in less than 3 minutes on a desktop computer.

Imaging Systems for Size Measurements of Debrisat Fragments

NASA Technical Reports Server (NTRS)

Shiotani, B.; Scruggs, T.; Toledo, R.; Fitz-Coy, N.; Liou, J. C.; Sorge, M.; Huynh, T.; Opiela, J.; Krisko, P.; Cowardin, H.

2017-01-01

The overall objective of the DebriSat project is to provide data to update existing standard spacecraft breakup models. One of the key sets of parameters used in these models is the physical dimensions of the fragments (i.e., length, average-cross sectional area, and volume). For the DebriSat project, only fragments with at least one dimension greater than 2 mm are collected and processed. Additionally, a significant portion of the fragments recovered from the impact test are needle-like and/or flat plate-like fragments where their heights are almost negligible in comparison to their other dimensions. As a result, two fragment size categories were defined: 2D objects and 3D objects. While measurement systems are commercially available, factors such as measurement rates, system adaptability, size characterization limitations and equipment costs presented significant challenges to the project and a decision was made to develop our own size characterization systems. The size characterization systems consist of two automated image systems, one referred to as the 3D imaging system and the other as the 2D imaging system. Which imaging system to use depends on the classification of the fragment being measured. Both imaging systems utilize point-and-shoot cameras for object image acquisition and create representative point clouds of the fragments. The 3D imaging system utilizes a space-carving algorithm to generate a 3D point cloud, while the 2D imaging system utilizes an edge detection algorithm to generate a 2D point cloud. From the point clouds, the three largest orthogonal dimensions are determined using a convex hull algorithm. For 3D objects, in addition to the three largest orthogonal dimensions, the volume is computed via an alpha-shape algorithm applied to the point clouds. The average cross-sectional area is also computed for 3D objects. Both imaging systems have automated size measurements (image acquisition and image processing) driven by the need to quickly and accurately measure tens of thousands of debris fragments. Moreover, the automated size measurement reduces potential fragment damage/mishandling and ability for accuracy and repeatability. As the fragment characterization progressed, it became evident that the imaging systems had to be revised. For example, an additional view was added to the 2D imaging system to capture the height of the 2D object. This paper presents the DebriSat project's imaging systems and calculation techniques in detail; from design and development to maturation. The experiences and challenges are also shared.

Evaluation of Multiple Kernel Learning Algorithms for Crop Mapping Using Satellite Image Time-Series Data

NASA Astrophysics Data System (ADS)

Niazmardi, S.; Safari, A.; Homayouni, S.

2017-09-01

Crop mapping through classification of Satellite Image Time-Series (SITS) data can provide very valuable information for several agricultural applications, such as crop monitoring, yield estimation, and crop inventory. However, the SITS data classification is not straightforward. Because different images of a SITS data have different levels of information regarding the classification problems. Moreover, the SITS data is a four-dimensional data that cannot be classified using the conventional classification algorithms. To address these issues in this paper, we presented a classification strategy based on Multiple Kernel Learning (MKL) algorithms for SITS data classification. In this strategy, initially different kernels are constructed from different images of the SITS data and then they are combined into a composite kernel using the MKL algorithms. The composite kernel, once constructed, can be used for the classification of the data using the kernel-based classification algorithms. We compared the computational time and the classification performances of the proposed classification strategy using different MKL algorithms for the purpose of crop mapping. The considered MKL algorithms are: MKL-Sum, SimpleMKL, LPMKL and Group-Lasso MKL algorithms. The experimental tests of the proposed strategy on two SITS data sets, acquired by SPOT satellite sensors, showed that this strategy was able to provide better performances when compared to the standard classification algorithm. The results also showed that the optimization method of the used MKL algorithms affects both the computational time and classification accuracy of this strategy.

A curvature-based weighted fuzzy c-means algorithm for point clouds de-noising

NASA Astrophysics Data System (ADS)

Cui, Xin; Li, Shipeng; Yan, Xiutian; He, Xinhua

2018-04-01

In order to remove the noise of three-dimensional scattered point cloud and smooth the data without damnify the sharp geometric feature simultaneity, a novel algorithm is proposed in this paper. The feature-preserving weight is added to fuzzy c-means algorithm which invented a curvature weighted fuzzy c-means clustering algorithm. Firstly, the large-scale outliers are removed by the statistics of r radius neighboring points. Then, the algorithm estimates the curvature of the point cloud data by using conicoid parabolic fitting method and calculates the curvature feature value. Finally, the proposed clustering algorithm is adapted to calculate the weighted cluster centers. The cluster centers are regarded as the new points. The experimental results show that this approach is efficient to different scale and intensities of noise in point cloud with a high precision, and perform a feature-preserving nature at the same time. Also it is robust enough to different noise model.

The ESA Cloud CCI project: Generation of Multi Sensor consistent Cloud Properties with an Optimal Estimation Based Retrieval Algorithm

NASA Astrophysics Data System (ADS)

Jerg, M.; Stengel, M.; Hollmann, R.; Poulsen, C.

2012-04-01

The ultimate objective of the ESA Climate Change Initiative (CCI) Cloud project is to provide long-term coherent cloud property data sets exploiting and improving on the synergetic capabilities of past, existing, and upcoming European and American satellite missions. The synergetic approach allows not only for improved accuracy and extended temporal and spatial sampling of retrieved cloud properties better than those provided by single instruments alone but potentially also for improved (inter-)calibration and enhanced homogeneity and stability of the derived time series. Such advances are required by the scientific community to facilitate further progress in satellite-based climate monitoring, which leads to a better understanding of climate. Some of the primary objectives of ESA Cloud CCI Cloud are (1) the development of inter-calibrated radiance data sets, so called Fundamental Climate Data Records - for ESA and non ESA instruments through an international collaboration, (2) the development of an optimal estimation based retrieval framework for cloud related essential climate variables like cloud cover, cloud top height and temperature, liquid and ice water path, and (3) the development of two multi-annual global data sets for the mentioned cloud properties including uncertainty estimates. These two data sets are characterized by different combinations of satellite systems: the AVHRR heritage product comprising (A)ATSR, AVHRR and MODIS and the novel (A)ATSR - MERIS product which is based on a synergetic retrieval using both instruments. Both datasets cover the years 2007-2009 in the first project phase. ESA Cloud CCI will also carry out a comprehensive validation of the cloud property products and provide a common data base as in the framework of the Global Energy and Water Cycle Experiment (GEWEX). The presentation will give an overview of the ESA Cloud CCI project and its goals and approaches and then continue with results from the Round Robin algorithm comparison exercise carried out at the beginning of the project which included three algorithms. The purpose of the exercise was to assess and compare existing cloud retrieval algorithms in order to chose one of them as backbone of the retrieval system and also identify areas of potential improvement and general strengths and weaknesses of the algorithm. Furthermore the presentation will elaborate on the optimal estimation algorithm subsequently chosen to derive the heritage product and which is presently further developed and will be employed for the AVHRR heritage product. The algorithm's capabilities to coherently and simultaneously process all radiative input and yield retrieval parameters together with associated uncertainty estimates will be presented together with first results for the heritage product. In the course of the project the algorithm is being developed into a freely and publicly available community retrieval system for interested scientists.

Global Single and Multiple Cloud Classification with a Fuzzy Logic Expert System

NASA Technical Reports Server (NTRS)

Welch, Ronald M.; Tovinkere, Vasanth; Titlow, James; Baum, Bryan A.

1996-01-01

An unresolved problem in remote sensing concerns the analysis of satellite imagery containing both single and multiple cloud layers. While cloud parameterizations are very important both in global climate models and in studies of the Earth's radiation budget, most cloud retrieval schemes, such as the bispectral method used by the International Satellite Cloud Climatology Project (ISCCP), have no way of determining whether overlapping cloud layers exist in any group of satellite pixels. Coakley (1983) used a spatial coherence method to determine whether a region contained more than one cloud layer. Baum et al. (1995) developed a scheme for detection and analysis of daytime multiple cloud layers using merged AVHRR (Advanced Very High Resolution Radiometer) and HIRS (High-resolution Infrared Radiometer Sounder) data collected during the First ISCCP Regional Experiment (FIRE) Cirrus 2 field campaign. Baum et al. (1995) explored the use of a cloud classification technique based on AVHRR data. This study examines the feasibility of applying the cloud classifier to global satellite imagery.

Informal settlement classification using point-cloud and image-based features from UAV data

NASA Astrophysics Data System (ADS)

Gevaert, C. M.; Persello, C.; Sliuzas, R.; Vosselman, G.

2017-03-01

Unmanned Aerial Vehicles (UAVs) are capable of providing very high resolution and up-to-date information to support informal settlement upgrading projects. In order to provide accurate basemaps, urban scene understanding through the identification and classification of buildings and terrain is imperative. However, common characteristics of informal settlements such as small, irregular buildings with heterogeneous roof material and large presence of clutter challenge state-of-the-art algorithms. Furthermore, it is of interest to analyse which fundamental attributes are suitable for describing these objects in different geographic locations. This work investigates how 2D radiometric and textural features, 2.5D topographic features, and 3D geometric features obtained from UAV imagery can be integrated to obtain a high classification accuracy in challenging classification problems for the analysis of informal settlements. UAV datasets from informal settlements in two different countries are compared in order to identify salient features for specific objects in heterogeneous urban environments. Findings show that the integration of 2D and 3D features leads to an overall accuracy of 91.6% and 95.2% respectively for informal settlements in Kigali, Rwanda and Maldonado, Uruguay.

Remote Sensing of Cloud Top Height from SEVIRI: Analysis of Eleven Current Retrieval Algorithms

NASA Technical Reports Server (NTRS)

Hamann, U.; Walther, A.; Baum, B.; Bennartz, R.; Bugliaro, L.; Derrien, M.; Francis, P. N.; Heidinger, A.; Joro, S.; Kniffka, A.;

Infrared Cloud Imager Development for Atmospheric Optical Communication Characterization, and Measurements at the JPL Table Mountain Facility

NASA Astrophysics Data System (ADS)

Nugent, P. W.; Shaw, J. A.; Piazzolla, S.

2013-02-01

The continuous demand for high data return in deep space and near-Earth satellite missions has led NASA and international institutions to consider alternative technologies for high-data-rate communications. One solution is the establishment of wide-bandwidth Earth-space optical communication links, which require (among other things) a nearly obstruction-free atmospheric path. Considering the atmospheric channel, the most common and most apparent impairments on Earth-space optical communication paths arise from clouds. Therefore, the characterization of the statistical behavior of cloud coverage for optical communication ground station candidate sites is of vital importance. In this article, we describe the development and deployment of a ground-based, long-wavelength infrared cloud imaging system able to monitor and characterize the cloud coverage. This system is based on a commercially available camera with a 62-deg diagonal field of view. A novel internal-shutter-based calibration technique allows radiometric calibration of the camera, which operates without a thermoelectric cooler. This cloud imaging system provides continuous day-night cloud detection with constant sensitivity. The cloud imaging system also includes data-processing algorithms that calculate and remove atmospheric emission to isolate cloud signatures, and enable classification of clouds according to their optical attenuation. Measurements of long-wavelength infrared cloud radiance are used to retrieve the optical attenuation (cloud optical depth due to absorption and scattering) in the wavelength range of interest from visible to near-infrared, where the cloud attenuation is quite constant. This article addresses the specifics of the operation, calibration, and data processing of the imaging system that was deployed at the NASA/JPL Table Mountain Facility (TMF) in California. Data are reported from July 2008 to July 2010. These data describe seasonal variability in cloud cover at the TMF site, with cloud amount (percentage of cloudy pixels) peaking at just over 51 percent during February, of which more than 60 percent had optical attenuation exceeding 12 dB at wavelengths in the range from the visible to the near-infrared. The lowest cloud amount was found during August, averaging 19.6 percent, and these clouds were mostly optically thin, with low attenuation.

Progress towards NASA MODIS and Suomi NPP Cloud Property Data Record Continuity

NASA Astrophysics Data System (ADS)

Platnick, S.; Meyer, K.; Holz, R.; Ackerman, S. A.; Heidinger, A.; Wind, G.; Platnick, S. E.; Wang, C.; Marchant, B.; Frey, R.

2017-12-01

The Suomi NPP VIIRS imager provides an opportunity to extend the 17+ year EOS MODIS climate data record into the next generation operational era. Similar to MODIS, VIIRS provides visible through IR observations at moderate spatial resolution with a 1330 LT equatorial crossing consistent with the MODIS on the Aqua platform. However, unlike MODIS, VIIRS lacks key water vapor and CO2 absorbing channels used for high cloud detection and cloud-top property retrievals. In addition, there is a significant mismatch in the spectral location of the 2.2 μm shortwave-infrared channels used for cloud optical/microphysical retrievals and cloud thermodynamic phase. Given these instrument differences between MODIS EOS and VIIRS S-NPP/JPSS, a merged MODIS-VIIRS cloud record to serve the science community in the coming decades requires different algorithm approaches than those used for MODIS alone. This new approach includes two parallel efforts: (1) Imager-only algorithms with only spectral channels common to VIIRS and MODIS (i.e., eliminate use of MODIS CO2 and NIR/IR water vapor channels). Since the algorithms are run with similar spectral observations, they provide a basis for establishing a continuous cloud data record across the two imagers. (2) Merged imager and sounder measurements (i.e.., MODIS-AIRS, VIIRS-CrIS) in lieu of higher-spatial resolution MODIS absorption channels absent on VIIRS. The MODIS-VIIRS continuity algorithm for cloud optical property retrievals leverages heritage algorithms that produce the existing MODIS cloud mask (MOD35), optical and microphysical properties product (MOD06), and the NOAA AWG Cloud Height Algorithm (ACHA). We discuss our progress towards merging the MODIS observational record with VIIRS in order to generate cloud optical property climate data record continuity across the observing systems. In addition, we summarize efforts to reconcile apparent radiometric biases between analogous imager channels, a critical consideration for obtaining inter-sensor climate data record continuity.

Classification of Parkinson's disease utilizing multi-edit nearest-neighbor and ensemble learning algorithms with speech samples.

PubMed

Zhang, He-Hua; Yang, Liuyang; Liu, Yuchuan; Wang, Pin; Yin, Jun; Li, Yongming; Qiu, Mingguo; Zhu, Xueru; Yan, Fang

2016-11-16

The use of speech based data in the classification of Parkinson disease (PD) has been shown to provide an effect, non-invasive mode of classification in recent years. Thus, there has been an increased interest in speech pattern analysis methods applicable to Parkinsonism for building predictive tele-diagnosis and tele-monitoring models. One of the obstacles in optimizing classifications is to reduce noise within the collected speech samples, thus ensuring better classification accuracy and stability. While the currently used methods are effect, the ability to invoke instance selection has been seldomly examined. In this study, a PD classification algorithm was proposed and examined that combines a multi-edit-nearest-neighbor (MENN) algorithm and an ensemble learning algorithm. First, the MENN algorithm is applied for selecting optimal training speech samples iteratively, thereby obtaining samples with high separability. Next, an ensemble learning algorithm, random forest (RF) or decorrelated neural network ensembles (DNNE), is used to generate trained samples from the collected training samples. Lastly, the trained ensemble learning algorithms are applied to the test samples for PD classification. This proposed method was examined using a more recently deposited public datasets and compared against other currently used algorithms for validation. Experimental results showed that the proposed algorithm obtained the highest degree of improved classification accuracy (29.44%) compared with the other algorithm that was examined. Furthermore, the MENN algorithm alone was found to improve classification accuracy by as much as 45.72%. Moreover, the proposed algorithm was found to exhibit a higher stability, particularly when combining the MENN and RF algorithms. This study showed that the proposed method could improve PD classification when using speech data and can be applied to future studies seeking to improve PD classification methods.

cOSPREY: A Cloud-Based Distributed Algorithm for Large-Scale Computational Protein Design

PubMed Central

Pan, Yuchao; Dong, Yuxi; Zhou, Jingtian; Hallen, Mark; Donald, Bruce R.; Xu, Wei

2016-01-01

Abstract Finding the global minimum energy conformation (GMEC) of a huge combinatorial search space is the key challenge in computational protein design (CPD) problems. Traditional algorithms lack a scalable and efficient distributed design scheme, preventing researchers from taking full advantage of current cloud infrastructures. We design cloud OSPREY (cOSPREY), an extension to a widely used protein design software OSPREY, to allow the original design framework to scale to the commercial cloud infrastructures. We propose several novel designs to integrate both algorithm and system optimizations, such as GMEC-specific pruning, state search partitioning, asynchronous algorithm state sharing, and fault tolerance. We evaluate cOSPREY on three different cloud platforms using different technologies and show that it can solve a number of large-scale protein design problems that have not been possible with previous approaches. PMID:27154509

Partial Storage Optimization and Load Control Strategy of Cloud Data Centers

PubMed Central

2015-01-01

We present a novel approach to solve the cloud storage issues and provide a fast load balancing algorithm. Our approach is based on partitioning and concurrent dual direction download of the files from multiple cloud nodes. Partitions of the files are saved on the cloud rather than the full files, which provide a good optimization to the cloud storage usage. Only partial replication is used in this algorithm to ensure the reliability and availability of the data. Our focus is to improve the performance and optimize the storage usage by providing the DaaS on the cloud. This algorithm solves the problem of having to fully replicate large data sets, which uses up a lot of precious space on the cloud nodes. Reducing the space needed will help in reducing the cost of providing such space. Moreover, performance is also increased since multiple cloud servers will collaborate to provide the data to the cloud clients in a faster manner. PMID:25973444

Partial storage optimization and load control strategy of cloud data centers.

PubMed

Al Nuaimi, Klaithem; Mohamed, Nader; Al Nuaimi, Mariam; Al-Jaroodi, Jameela

2015-01-01

We present a novel approach to solve the cloud storage issues and provide a fast load balancing algorithm. Our approach is based on partitioning and concurrent dual direction download of the files from multiple cloud nodes. Partitions of the files are saved on the cloud rather than the full files, which provide a good optimization to the cloud storage usage. Only partial replication is used in this algorithm to ensure the reliability and availability of the data. Our focus is to improve the performance and optimize the storage usage by providing the DaaS on the cloud. This algorithm solves the problem of having to fully replicate large data sets, which uses up a lot of precious space on the cloud nodes. Reducing the space needed will help in reducing the cost of providing such space. Moreover, performance is also increased since multiple cloud servers will collaborate to provide the data to the cloud clients in a faster manner.

Operational multisensor sea ice concentration algorithm utilizing Sentinel-1 and AMSR2 data

NASA Astrophysics Data System (ADS)

Dinessen, Frode

2017-04-01

The Norwegian Ice Service provide ice charts of the European part of the Arctic every weekday. The charts are produced from a manually interpretation of satellite data where SAR (Synthetic Aperture Radar) data plays a central role because of its high spatial resolution and Independence of cloud cover. A new chart is produced every weekday and the charts are distributed through the CMEMS portal. After the launch of Sentinel-1A and B the number of available SAR data have significant increased making it difficult to utilize all the data in a manually process. This in combination with a user demand for a more frequent update of the ice conditions, also during the weekends, have made it important to focus the development on utilizing the high resolution Sentinel-1 data in an automatic sea ice concentration analysis. The algorithm developed here is based on a multi sensor approach using an optimal interpolation to combine sea ice concentration products derived from Sentinel-1 and passive microwave data from AMSR2. The Sentinel-1 data is classified with a Bayesian SAR classification algorithm using data in extra wide mode dual polarization (HH/HV) to separate ice and water in the full 40x40 meter spatial resolution. From the classification of ice/water the sea ice concentration is estimated by calculating amount of ice within an area of 1x1 km. The AMSR2 sea ice concentration are produced as part of the EUMETSAT Ocean and Sea Ice Satellite Application Facility (OSI SAF) project and utilize the 89 GHz channel to produce a concentration product with a 3km spatial resolution. Results from the automatic classification will be presented.

The Validation of Cloud Retrieval Algorithms Using Synthetic Datasets

NASA Astrophysics Data System (ADS)

Kokhanovsky, Alexander; Fischer, Jurgen; Linstrot, Rasmus; Meirink, Jan Fokke; Poulsen, Caroline; Preusker, Rene; Siddans, Richard; Thomas, Gareth; Arnold, Chris; Grainger, Roy; Lilli, Luca; Rozanov, Vladimir

2012-11-01

We have performed the inter-comparison study of cloud property retrievals using algorithms initially developed for AATSR (ORAC, RAL-Oxford University), AVHRR and SEVIRI (CPP, KNMI), SCIAMACHY/GOME (SACURA, University of Bremen), and MERIS (ANNA, Free University of Berlin). The accuracy of retrievals of cloud optical thickness (COT), effective radius (ER) of droplets, and cloud top height (CTH) is discussed.

MPLNET V3 Cloud and Planetary Boundary Layer Detection

NASA Technical Reports Server (NTRS)

Lewis, Jasper R.; Welton, Ellsworth J.; Campbell, James R.; Haftings, Phillip C.

2016-01-01

The NASA Micropulse Lidar Network Version 3 algorithms for planetary boundary layer and cloud detection are described and differences relative to the previous Version 2 algorithms are highlighted. A year of data from the Goddard Space Flight Center site in Greenbelt, MD consisting of diurnal and seasonal trends is used to demonstrate the results. Both the planetary boundary layer and cloud algorithms show significant improvement of the previous version.

Assessment of Uncertainty in Cloud Radiative Effects and Heating Rates through Retrieval Algorithm Differences: Analysis using 3-years of ARM data at Darwin, Australia

DOE Office of Scientific and Technical Information (OSTI.GOV)

Comstock, Jennifer M.; Protat, Alain; McFarlane, Sally A.

2013-05-22

Ground-based radar and lidar observations obtained at the Department of Energy’s Atmospheric Radiation Measurement Program’s Tropical Western Pacific site located in Darwin, Australia are used to retrieve ice cloud properties in anvil and cirrus clouds. Cloud microphysical properties derived from four different retrieval algorithms (two radar-lidar and two radar only algorithms) are compared by examining mean profiles and probability density functions of effective radius (Re), ice water content (IWC), extinction, ice number concentration, ice crystal fall speed, and vertical air velocity. Retrieval algorithm uncertainty is quantified using radiative flux closure exercises. The effect of uncertainty in retrieved quantities on themore » cloud radiative effect and radiative heating rates are presented. Our analysis shows that IWC compares well among algorithms, but Re shows significant discrepancies, which is attributed primarily to assumptions of particle shape. Uncertainty in Re and IWC translates into sometimes-large differences in cloud radiative effect (CRE) though the majority of cases have a CRE difference of roughly 10 W m-2 on average. These differences, which we believe are primarily driven by the uncertainty in Re, can cause up to 2 K/day difference in the radiative heating rates between algorithms.« less

A proposed study of multiple scattering through clouds up to 1 THz

NASA Technical Reports Server (NTRS)

Gerace, G. C.; Smith, E. K.

1992-01-01

A rigorous computation of the electromagnetic field scattered from an atmospheric liquid water cloud is proposed. The recent development of a fast recursive algorithm (Chew algorithm) for computing the fields scattered from numerous scatterers now makes a rigorous computation feasible. A method is presented for adapting this algorithm to a general case where there are an extremely large number of scatterers. It is also proposed to extend a new binary PAM channel coding technique (El-Khamy coding) to multiple levels with non-square pulse shapes. The Chew algorithm can be used to compute the transfer function of a cloud channel. Then the transfer function can be used to design an optimum El-Khamy code. In principle, these concepts can be applied directly to the realistic case of a time-varying cloud (adaptive channel coding and adaptive equalization). A brief review is included of some preliminary work on cloud dispersive effects on digital communication signals and on cloud liquid water spectra and correlations.

An Efficient Virtual Machine Consolidation Scheme for Multimedia Cloud Computing.

PubMed

Han, Guangjie; Que, Wenhui; Jia, Gangyong; Shu, Lei

2016-02-18

Cloud computing has innovated the IT industry in recent years, as it can delivery subscription-based services to users in the pay-as-you-go model. Meanwhile, multimedia cloud computing is emerging based on cloud computing to provide a variety of media services on the Internet. However, with the growing popularity of multimedia cloud computing, its large energy consumption cannot only contribute to greenhouse gas emissions, but also result in the rising of cloud users' costs. Therefore, the multimedia cloud providers should try to minimize its energy consumption as much as possible while satisfying the consumers' resource requirements and guaranteeing quality of service (QoS). In this paper, we have proposed a remaining utilization-aware (RUA) algorithm for virtual machine (VM) placement, and a power-aware algorithm (PA) is proposed to find proper hosts to shut down for energy saving. These two algorithms have been combined and applied to cloud data centers for completing the process of VM consolidation. Simulation results have shown that there exists a trade-off between the cloud data center's energy consumption and service-level agreement (SLA) violations. Besides, the RUA algorithm is able to deal with variable workload to prevent hosts from overloading after VM placement and to reduce the SLA violations dramatically.

An Efficient Virtual Machine Consolidation Scheme for Multimedia Cloud Computing

PubMed Central

Han, Guangjie; Que, Wenhui; Jia, Gangyong; Shu, Lei

2016-01-01

Cloud computing has innovated the IT industry in recent years, as it can delivery subscription-based services to users in the pay-as-you-go model. Meanwhile, multimedia cloud computing is emerging based on cloud computing to provide a variety of media services on the Internet. However, with the growing popularity of multimedia cloud computing, its large energy consumption cannot only contribute to greenhouse gas emissions, but also result in the rising of cloud users’ costs. Therefore, the multimedia cloud providers should try to minimize its energy consumption as much as possible while satisfying the consumers’ resource requirements and guaranteeing quality of service (QoS). In this paper, we have proposed a remaining utilization-aware (RUA) algorithm for virtual machine (VM) placement, and a power-aware algorithm (PA) is proposed to find proper hosts to shut down for energy saving. These two algorithms have been combined and applied to cloud data centers for completing the process of VM consolidation. Simulation results have shown that there exists a trade-off between the cloud data center’s energy consumption and service-level agreement (SLA) violations. Besides, the RUA algorithm is able to deal with variable workload to prevent hosts from overloading after VM placement and to reduce the SLA violations dramatically. PMID:26901201

Cloud Computing and Its Applications in GIS

NASA Astrophysics Data System (ADS)

Kang, Cao

2011-12-01

Cloud computing is a novel computing paradigm that offers highly scalable and highly available distributed computing services. The objectives of this research are to: 1. analyze and understand cloud computing and its potential for GIS; 2. discover the feasibilities of migrating truly spatial GIS algorithms to distributed computing infrastructures; 3. explore a solution to host and serve large volumes of raster GIS data efficiently and speedily. These objectives thus form the basis for three professional articles. The first article is entitled "Cloud Computing and Its Applications in GIS". This paper introduces the concept, structure, and features of cloud computing. Features of cloud computing such as scalability, parallelization, and high availability make it a very capable computing paradigm. Unlike High Performance Computing (HPC), cloud computing uses inexpensive commodity computers. The uniform administration systems in cloud computing make it easier to use than GRID computing. Potential advantages of cloud-based GIS systems such as lower barrier to entry are consequently presented. Three cloud-based GIS system architectures are proposed: public cloud- based GIS systems, private cloud-based GIS systems and hybrid cloud-based GIS systems. Public cloud-based GIS systems provide the lowest entry barriers for users among these three architectures, but their advantages are offset by data security and privacy related issues. Private cloud-based GIS systems provide the best data protection, though they have the highest entry barriers. Hybrid cloud-based GIS systems provide a compromise between these extremes. The second article is entitled "A cloud computing algorithm for the calculation of Euclidian distance for raster GIS". Euclidean distance is a truly spatial GIS algorithm. Classical algorithms such as the pushbroom and growth ring techniques require computational propagation through the entire raster image, which makes it incompatible with the distributed nature of cloud computing. This paper presents a parallel Euclidean distance algorithm that works seamlessly with the distributed nature of cloud computing infrastructures. The mechanism of this algorithm is to subdivide a raster image into sub-images and wrap them with a one pixel deep edge layer of individually computed distance information. Each sub-image is then processed by a separate node, after which the resulting sub-images are reassembled into the final output. It is shown that while any rectangular sub-image shape can be used, those approximating squares are computationally optimal. This study also serves as a demonstration of this subdivide and layer-wrap strategy, which would enable the migration of many truly spatial GIS algorithms to cloud computing infrastructures. However, this research also indicates that certain spatial GIS algorithms such as cost distance cannot be migrated by adopting this mechanism, which presents significant challenges for the development of cloud-based GIS systems. The third article is entitled "A Distributed Storage Schema for Cloud Computing based Raster GIS Systems". This paper proposes a NoSQL Database Management System (NDDBMS) based raster GIS data storage schema. NDDBMS has good scalability and is able to use distributed commodity computers, which make it superior to Relational Database Management Systems (RDBMS) in a cloud computing environment. In order to provide optimized data service performance, the proposed storage schema analyzes the nature of commonly used raster GIS data sets. It discriminates two categories of commonly used data sets, and then designs corresponding data storage models for both categories. As a result, the proposed storage schema is capable of hosting and serving enormous volumes of raster GIS data speedily and efficiently on cloud computing infrastructures. In addition, the scheme also takes advantage of the data compression characteristics of Quadtrees, thus promoting efficient data storage. Through this assessment of cloud computing technology, the exploration of the challenges and solutions to the migration of GIS algorithms to cloud computing infrastructures, and the examination of strategies for serving large amounts of GIS data in a cloud computing infrastructure, this dissertation lends support to the feasibility of building a cloud-based GIS system. However, there are still challenges that need to be addressed before a full-scale functional cloud-based GIS system can be successfully implemented. (Abstract shortened by UMI.)

Orbiting Carbon Observatory-2 (OCO-2) cloud screening algorithms; validation against collocated MODIS and CALIOP data

NASA Astrophysics Data System (ADS)

Taylor, T. E.; O'Dell, C. W.; Frankenberg, C.; Partain, P.; Cronk, H. Q.; Savtchenko, A.; Nelson, R. R.; Rosenthal, E. J.; Chang, A. Y.; Fisher, B.; Osterman, G.; Pollock, R. H.; Crisp, D.; Eldering, A.; Gunson, M. R.

2015-12-01

The objective of the National Aeronautics and Space Administration's (NASA) Orbiting Carbon Observatory-2 (OCO-2) mission is to retrieve the column-averaged carbon dioxide (CO2) dry air mole fraction (XCO2) from satellite measurements of reflected sunlight in the near-infrared. These estimates can be biased by clouds and aerosols within the instrument's field of view (FOV). Screening of the most contaminated soundings minimizes unnecessary calls to the computationally expensive Level 2 (L2) XCO2 retrieval algorithm. Hence, robust cloud screening methods have been an important focus of the OCO-2 algorithm development team. Two distinct, computationally inexpensive cloud screening algorithms have been developed for this application. The A-Band Preprocessor (ABP) retrieves the surface pressure using measurements in the 0.76 μm O2 A-band, neglecting scattering by clouds and aerosols, which introduce photon path-length (PPL) differences that can cause large deviations between the expected and retrieved surface pressure. The Iterative Maximum A-Posteriori (IMAP) Differential Optical Absorption Spectroscopy (DOAS) Preprocessor (IDP) retrieves independent estimates of the CO2 and H2O column abundances using observations taken at 1.61 μm (weak CO2 band) and 2.06 μm (strong CO2 band), while neglecting atmospheric scattering. The CO2 and H2O column abundances retrieved in these two spectral regions differ significantly in the presence of cloud and scattering aerosols. The combination of these two algorithms, which key off of different features in the spectra, provides the basis for cloud screening of the OCO-2 data set. To validate the OCO-2 cloud screening approach, collocated measurements from NASA's Moderate Resolution Imaging Spectrometer (MODIS), aboard the Aqua platform, were compared to results from the two OCO-2 cloud screening algorithms. With tuning to allow throughputs of ≃ 30 %, agreement between the OCO-2 and MODIS cloud screening methods is found to be ≃ 85 % over four 16-day orbit repeat cycles in both the winter (December) and spring (April-May) for OCO-2 nadir-land, glint-land and glint-water observations. No major, systematic, spatial or temporal dependencies were found, although slight differences in the seasonal data sets do exist and validation is more problematic with increasing solar zenith angle and when surfaces are covered in snow and ice and have complex topography. To further analyze the performance of the cloud screening algorithms, an initial comparison of OCO-2 observations was made to collocated measurements from the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) aboard the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO). These comparisons highlight the strength of the OCO-2 cloud screening algorithms in identifying high, thin clouds but suggest some difficulty in identifying some clouds near the surface, even when the optical thicknesses are greater than 1.

Hyperspectral feature mapping classification based on mathematical morphology

NASA Astrophysics Data System (ADS)

Liu, Chang; Li, Junwei; Wang, Guangping; Wu, Jingli

2016-03-01

This paper proposed a hyperspectral feature mapping classification algorithm based on mathematical morphology. Without the priori information such as spectral library etc., the spectral and spatial information can be used to realize the hyperspectral feature mapping classification. The mathematical morphological erosion and dilation operations are performed respectively to extract endmembers. The spectral feature mapping algorithm is used to carry on hyperspectral image classification. The hyperspectral image collected by AVIRIS is applied to evaluate the proposed algorithm. The proposed algorithm is compared with minimum Euclidean distance mapping algorithm, minimum Mahalanobis distance mapping algorithm, SAM algorithm and binary encoding mapping algorithm. From the results of the experiments, it is illuminated that the proposed algorithm's performance is better than that of the other algorithms under the same condition and has higher classification accuracy.

Remote sensing of cirrus cloud vertical size profile using MODIS data

NASA Astrophysics Data System (ADS)

Wang, Xingjuan; Liou, K. N.; Ou, Steve S. C.; Mace, G. G.; Deng, M.

2009-05-01

This paper describes an algorithm for inferring cirrus cloud top and cloud base effective particle sizes and cloud optical thickness from the Moderate Resolution Imaging Spectroradiometer (MODIS) 0.645, 1.64 and 2.13, and 3.75 μm band reflectances/radiances. This approach uses a successive minimization method based on a look-up library of precomputed reflectances/radiances from an adding-doubling radiative transfer program, subject to corrections for Rayleigh scattering at the 0.645 μm band, above-cloud water vapor absorption, and 3.75 μm thermal emission. The algorithmic accuracy and limitation of the retrieval method were investigated by synthetic retrievals subject to the instrument noise and the perturbation of input parameters. The retrieval algorithm was applied to three MODIS cirrus scenes over the Atmospheric Radiation Measurement Program's southern Great Plain site, north central China, and northeast Asia. The reliability of retrieved cloud optical thicknesses and mean effective particle sizes was evaluated by comparison with MODIS cloud products and qualitatively good correlations were obtained for all three cases, indicating that the performance of the vertical sizing algorithm is comparable with the MODIS retrieval program. Retrieved cloud top and cloud base ice crystal effective sizes were also compared with those derived from the collocated ground-based millimeter wavelength cloud radar for the first case and from the Cloud Profiling Radar onboard CloudSat for the other two cases. Differences between retrieved and radar-derived cloud properties are discussed in light of assumptions made in the collocation process and limitations in radar remote sensing characteristics.

Optical Algorithm for Cloud Shadow Detection Over Water

DTIC Science & Technology

2013-02-01

REPORT DATE (DD-MM-YYYY) 05-02-2013 2. REPORT TYPE Journal Article 3. DATES COVERED (From ■ To) 4. TITLE AND SUBTITLE Optical Algorithm for Cloud...particularly over humid tropical regions. Throughout the year, about two-thirds of the Earth’s surface is always covered by clouds [1]. The problem...V. Khlopenkov and A. P. Trishchenko, "SPARC: New cloud, snow , cloud shadow detection scheme for historical I-km AVHHR data over Canada," / Atmos

Improvements to the CERES Cloud Detection Algorithm using Himawari 8 Data and Validation using CALIPSO and CATS Lidar Observations

NASA Astrophysics Data System (ADS)

Trepte, Q.; Minnis, P.; Palikonda, R.; Yost, C. R.; Rodier, S. D.; Trepte, C. R.; McGill, M. J.

2016-12-01

Geostationary satellites provide continuous cloud and meteorological observations important for weather forecasting and for understanding climate processes. The Himawari-8 satellite represents a new generation of measurement capabilities with significantly improved resolution and enhanced spectral information. The satellite was launched in October 2014 by the Japanese Meteorological Agency and is centered at 140° E to provide coverage over eastern Asia and the western Pacific region. A cloud detection algorithm was developed as part of the CERES Cloud Mask algorithm using the Advanced Himawari Imager (AHI), a 16 channel multi-spectral imager. The algorithm was originally designed for use with Meteosat Second Generation (MSG) data and has been adapted for Himawari-8 AHI measurements. This paper will describe the improvements in the Himawari cloud mask including daytime ocean low cloud and aerosol discrimination, nighttime thin cirrus detection, and Australian desert and coastal cloud detection. The statistics from matched CERES Himawari cloud mask results with CALIPSO lidar data and with new observations from the CATS lidar will also be presented. A feature of the CATS instrument on board the International Space Station is that it gives information at different solar viewing times to examine the diurnal variation of clouds and this provides an ability to evaluate the performance of the cloud mask for different sun angles.

Intelligent cloud computing security using genetic algorithm as a computational tools

NASA Astrophysics Data System (ADS)

Razuky AL-Shaikhly, Mazin H.

2018-05-01

An essential change had occurred in the field of Information Technology which represented with cloud computing, cloud giving virtual assets by means of web yet awesome difficulties in the field of information security and security assurance. Currently main problem with cloud computing is how to improve privacy and security for cloud “cloud is critical security”. This paper attempts to solve cloud security by using intelligent system with genetic algorithm as wall to provide cloud data secure, all services provided by cloud must detect who receive and register it to create list of users (trusted or un-trusted) depend on behavior. The execution of present proposal has shown great outcome.

LSAH: a fast and efficient local surface feature for point cloud registration

NASA Astrophysics Data System (ADS)

Lu, Rongrong; Zhu, Feng; Wu, Qingxiao; Kong, Yanzi

2018-04-01

Point cloud registration is a fundamental task in high level three dimensional applications. Noise, uneven point density and varying point cloud resolutions are the three main challenges for point cloud registration. In this paper, we design a robust and compact local surface descriptor called Local Surface Angles Histogram (LSAH) and propose an effectively coarse to fine algorithm for point cloud registration. The LSAH descriptor is formed by concatenating five normalized sub-histograms into one histogram. The five sub-histograms are created by accumulating a different type of angle from a local surface patch respectively. The experimental results show that our LSAH is more robust to uneven point density and point cloud resolutions than four state-of-the-art local descriptors in terms of feature matching. Moreover, we tested our LSAH based coarse to fine algorithm for point cloud registration. The experimental results demonstrate that our algorithm is robust and efficient as well.

Summary of along-track data from the earth radiation budget satellite for several representative ocean regions

NASA Technical Reports Server (NTRS)

Brooks, David R.; Fenn, Marta A.

1988-01-01

For several days in January and August 1985, the Earth Radiation Budget Satellite, a component of the Earth Radiation Budget Experiment (ERBE), was operated in an along-track scanning mode. A survey of radiance measurements taken in this mode is given for five ocean regions: the north and south Atlantic, the Arabian Sea, the western Pacific north of the Equator, and part of the Intertropical Convergence Zone. Each overflight contains information about the clear scene and three cloud categories: partly cloudy, mostly cloudy, and overcast. The data presented include the variation of longwave and shortwave radiance in each scene classification as a function of viewing zenity angle during each overflight of one of the five target regions. Several features of interest in the development of anisotropic models are evident, including the azimuthal dependence of shortwave radiance that is an essential feature of shortwave bidirectional models. The data also demonstrate that the scene classification algorithm employed by the ERBE results in scene classifications that are a function of viewing geometry.

Robust point cloud classification based on multi-level semantic relationships for urban scenes

NASA Astrophysics Data System (ADS)

Zhu, Qing; Li, Yuan; Hu, Han; Wu, Bo

2017-07-01

The semantic classification of point clouds is a fundamental part of three-dimensional urban reconstruction. For datasets with high spatial resolution but significantly more noises, a general trend is to exploit more contexture information to surmount the decrease of discrimination of features for classification. However, previous works on adoption of contexture information are either too restrictive or only in a small region and in this paper, we propose a point cloud classification method based on multi-level semantic relationships, including point-homogeneity, supervoxel-adjacency and class-knowledge constraints, which is more versatile and incrementally propagate the classification cues from individual points to the object level and formulate them as a graphical model. The point-homogeneity constraint clusters points with similar geometric and radiometric properties into regular-shaped supervoxels that correspond to the vertices in the graphical model. The supervoxel-adjacency constraint contributes to the pairwise interactions by providing explicit adjacent relationships between supervoxels. The class-knowledge constraint operates at the object level based on semantic rules, guaranteeing the classification correctness of supervoxel clusters at that level. International Society of Photogrammetry and Remote Sensing (ISPRS) benchmark tests have shown that the proposed method achieves state-of-the-art performance with an average per-area completeness and correctness of 93.88% and 95.78%, respectively. The evaluation of classification of photogrammetric point clouds and DSM generated from aerial imagery confirms the method's reliability in several challenging urban scenes.

An Algorithm for the Retrieval of Droplet Number Concentration and Geometrical Thickness of Stratiform Marine Boundary Layer Clouds Applied to MODIS Radiometric Observations.

NASA Astrophysics Data System (ADS)

Schüller, Lothar; Bennartz, Ralf; Fischer, Jürgen; Brenguier, Jean-Louis

2005-01-01

Algorithms are now currently used for the retrieval of cloud optical thickness and droplet effective radius from multispectral radiance measurements. This paper extends their application to the retrieval of cloud droplet number concentration, cloud geometrical thickness, and liquid water path in shallow convective clouds, using an algorithm that was previously tested with airborne measurements of cloud radiances and validated against in situ measurements of the same clouds. The retrieval is based on a stratified cloud model of liquid water content and droplet spectrum. Radiance measurements in visible and near-infrared channels of the Moderate Resolution Imaging Spectroradiometer (MODIS), which is operated from the NASA platforms Terra and Aqua, are analyzed. Because of uncertainties in the simulation of the continental surface reflectance, the algorithm is presently limited to the monitoring of the microphysical structure of boundary layer clouds over the ocean. Two MODIS scenes of extended cloud fields over the North Atlantic Ocean trade wind region are processed. A transport and dispersion model (the Hybrid Single-Particle Lagrangian Integrated Trajectory Model, HYSPLIT4) is also used to characterize the origin of the air masses and hence their aerosol regimes. One cloud field formed in an air mass that was advected from southern Europe and North Africa. It shows high values of the droplet concentration when compared with the second cloud system, which developed in a more pristine environment. The more pristine case also exhibits a higher geometrical thickness and, thus, liquid water path, which counterbalances the expected cloud albedo increase of the polluted case. Estimates of cloud liquid water path are then compared with retrievals from the Special Sensor Microwave Imager (SSM/I). SSM/I-derived liquid water paths are in good agreement with the MODIS-derived values.

The problem of regime summaries of the data from radar observations. [for cloud system identification

NASA Technical Reports Server (NTRS)

Divinskaya, B. S.; Salman, Y. M.

1975-01-01

Peculiarities of the radar information about clouds are examined in comparison with visual data. An objective radar classification is presented and the relation of it to the meteorological classification is shown. The advisability of storage and summarization of the primary radar data for regime purposes is substantiated.

Interactive Classification of Construction Materials: Feedback Driven Framework for Annotation and Analysis of 3d Point Clouds

NASA Astrophysics Data System (ADS)

Hess, M. R.; Petrovic, V.; Kuester, F.

2017-08-01

Digital documentation of cultural heritage structures is increasingly more common through the application of different imaging techniques. Many works have focused on the application of laser scanning and photogrammetry techniques for the acquisition of threedimensional (3D) geometry detailing cultural heritage sites and structures. With an abundance of these 3D data assets, there must be a digital environment where these data can be visualized and analyzed. Presented here is a feedback driven visualization framework that seamlessly enables interactive exploration and manipulation of massive point cloud data. The focus of this work is on the classification of different building materials with the goal of building more accurate as-built information models of historical structures. User defined functions have been tested within the interactive point cloud visualization framework to evaluate automated and semi-automated classification of 3D point data. These functions include decisions based on observed color, laser intensity, normal vector or local surface geometry. Multiple case studies are presented here to demonstrate the flexibility and utility of the presented point cloud visualization framework to achieve classification objectives.

Dynamic virtual machine allocation policy in cloud computing complying with service level agreement using CloudSim

NASA Astrophysics Data System (ADS)

Aneri, Parikh; Sumathy, S.

2017-11-01

Cloud computing provides services over the internet and provides application resources and data to the users based on their demand. Base of the Cloud Computing is consumer provider model. Cloud provider provides resources which consumer can access using cloud computing model in order to build their application based on their demand. Cloud data center is a bulk of resources on shared pool architecture for cloud user to access. Virtualization is the heart of the Cloud computing model, it provides virtual machine as per application specific configuration and those applications are free to choose their own configuration. On one hand, there is huge number of resources and on other hand it has to serve huge number of requests effectively. Therefore, resource allocation policy and scheduling policy play very important role in allocation and managing resources in this cloud computing model. This paper proposes the load balancing policy using Hungarian algorithm. Hungarian Algorithm provides dynamic load balancing policy with a monitor component. Monitor component helps to increase cloud resource utilization by managing the Hungarian algorithm by monitoring its state and altering its state based on artificial intelligent. CloudSim used in this proposal is an extensible toolkit and it simulates cloud computing environment.

Whole Sky Imager Characterization of Sky Obscuration by Clouds for the Starfire Optical Range

DTIC Science & Technology

2010-01-11

9.3 Further Algorithm Development and Evaluation 58 9.4 Analysis of the Data Base 58 10.0 DISCUSSION OF CONTRACT REQUIREMENTS 59 10.1...clouds, Site 5 Feb 14 2008 0900 28 21 Transmittance map, Moonlight , clear sky, Site 5 Feb 3 2008 0700 28 22 Transmittance map, Moonlight , thin...clouds, Site 5 Feb 8 2008 1200 29 23 Transmittance map, Moonlight , broken clouds, Site 5 Feb 2 2008 0800 29 24 Cloud Algorithm Results, Moonlight

Integrated Change Detection and Classification in Urban Areas Based on Airborne Laser Scanning Point Clouds.

PubMed

Tran, Thi Huong Giang; Ressl, Camillo; Pfeifer, Norbert

2018-02-03

This paper suggests a new approach for change detection (CD) in 3D point clouds. It combines classification and CD in one step using machine learning. The point cloud data of both epochs are merged for computing features of four types: features describing the point distribution, a feature relating to relative terrain elevation, features specific for the multi-target capability of laser scanning, and features combining the point clouds of both epochs to identify the change. All these features are merged in the points and then training samples are acquired to create the model for supervised classification, which is then applied to the whole study area. The final results reach an overall accuracy of over 90% for both epochs of eight classes: lost tree, new tree, lost building, new building, changed ground, unchanged building, unchanged tree, and unchanged ground.

a Two-Step Classification Approach to Distinguishing Similar Objects in Mobile LIDAR Point Clouds

NASA Astrophysics Data System (ADS)

He, H.; Khoshelham, K.; Fraser, C.

2017-09-01

Nowadays, lidar is widely used in cultural heritage documentation, urban modeling, and driverless car technology for its fast and accurate 3D scanning ability. However, full exploitation of the potential of point cloud data for efficient and automatic object recognition remains elusive. Recently, feature-based methods have become very popular in object recognition on account of their good performance in capturing object details. Compared with global features describing the whole shape of the object, local features recording the fractional details are more discriminative and are applicable for object classes with considerable similarity. In this paper, we propose a two-step classification approach based on point feature histograms and the bag-of-features method for automatic recognition of similar objects in mobile lidar point clouds. Lamp post, street light and traffic sign are grouped as one category in the first-step classification for their inter similarity compared with tree and vehicle. A finer classification of the lamp post, street light and traffic sign based on the result of the first-step classification is implemented in the second step. The proposed two-step classification approach is shown to yield a considerable improvement over the conventional one-step classification approach.

Waves on Ice

Atmospheric Science Data Center

2013-04-16

article title:  Waves on White: Ice or Clouds?     View Larger ... like a wavy cloud pattern was actually a wavy pattern on the ice surface. One of MISR's cloud classification products, the Angular Signature ...

Performance comparison of heuristic algorithms for task scheduling in IaaS cloud computing environment.

PubMed

Madni, Syed Hamid Hussain; Abd Latiff, Muhammad Shafie; Abdullahi, Mohammed; Abdulhamid, Shafi'i Muhammad; Usman, Mohammed Joda

2017-01-01

Cloud computing infrastructure is suitable for meeting computational needs of large task sizes. Optimal scheduling of tasks in cloud computing environment has been proved to be an NP-complete problem, hence the need for the application of heuristic methods. Several heuristic algorithms have been developed and used in addressing this problem, but choosing the appropriate algorithm for solving task assignment problem of a particular nature is difficult since the methods are developed under different assumptions. Therefore, six rule based heuristic algorithms are implemented and used to schedule autonomous tasks in homogeneous and heterogeneous environments with the aim of comparing their performance in terms of cost, degree of imbalance, makespan and throughput. First Come First Serve (FCFS), Minimum Completion Time (MCT), Minimum Execution Time (MET), Max-min, Min-min and Sufferage are the heuristic algorithms considered for the performance comparison and analysis of task scheduling in cloud computing.

Performance comparison of heuristic algorithms for task scheduling in IaaS cloud computing environment

PubMed Central

Madni, Syed Hamid Hussain; Abd Latiff, Muhammad Shafie; Abdullahi, Mohammed; Usman, Mohammed Joda

2017-01-01

Cloud computing infrastructure is suitable for meeting computational needs of large task sizes. Optimal scheduling of tasks in cloud computing environment has been proved to be an NP-complete problem, hence the need for the application of heuristic methods. Several heuristic algorithms have been developed and used in addressing this problem, but choosing the appropriate algorithm for solving task assignment problem of a particular nature is difficult since the methods are developed under different assumptions. Therefore, six rule based heuristic algorithms are implemented and used to schedule autonomous tasks in homogeneous and heterogeneous environments with the aim of comparing their performance in terms of cost, degree of imbalance, makespan and throughput. First Come First Serve (FCFS), Minimum Completion Time (MCT), Minimum Execution Time (MET), Max-min, Min-min and Sufferage are the heuristic algorithms considered for the performance comparison and analysis of task scheduling in cloud computing. PMID:28467505

Study on Huizhou architecture of point cloud registration based on optimized ICP algorithm

NASA Astrophysics Data System (ADS)

Zhang, Runmei; Wu, Yulu; Zhang, Guangbin; Zhou, Wei; Tao, Yuqian

2018-03-01

In view of the current point cloud registration software has high hardware requirements, heavy workload and moltiple interactive definition, the source of software with better processing effect is not open, a two--step registration method based on normal vector distribution feature and coarse feature based iterative closest point (ICP) algorithm is proposed in this paper. This method combines fast point feature histogram (FPFH) algorithm, define the adjacency region of point cloud and the calculation model of the distribution of normal vectors, setting up the local coordinate system for each key point, and obtaining the transformation matrix to finish rough registration, the rough registration results of two stations are accurately registered by using the ICP algorithm. Experimental results show that, compared with the traditional ICP algorithm, the method used in this paper has obvious time and precision advantages for large amount of point clouds.

Comparison of the MODIS Multilayer Cloud Detection and Thermodynamic Phase Products with CALIPSO and CloudSat

NASA Technical Reports Server (NTRS)

Platnick, Steven; King, Michael D.; Wind, Gala; Holz, Robert E.; Ackerman, Steven A.; Nagle, Fred W.

2008-01-01

CALIPSO and CloudSat, launched in June 2006, provide global active remote sensing measurements of clouds and aerosols that can be used for validation of a variety of passive imager retrievals derived from instruments flying on the Aqua spacecraft and other A-Train platforms. The most recent processing effort for the MODIS Atmosphere Team, referred to as the "Collection 5" stream, includes a research-level multilayer cloud detection algorithm that uses both thermodynamic phase information derived from a combination of solar and thermal emission bands to discriminate layers of different phases, as well as true layer separation discrimination using a moderately absorbing water vapor band. The multilayer detection algorithm is designed to provide a means of assessing the applicability of 1D cloud models used in the MODIS cloud optical and microphysical product retrieval, which are generated at a 1 h resolution. Using pixel-level collocations of MODIS Aqua, CALIOP, and CloudSat radar measurements, we investigate the global performance of the thermodynamic phase and multilayer cloud detection algorithms.

Convective and Stratiform Precipitation Processes and their Relationship to Latent Heating

NASA Technical Reports Server (NTRS)

Tao, Wei-Kuo; Lang, Steve; Zeng, Xiping; Shige, Shoichi; Takayabu, Yukari

2009-01-01

The global hydrological cycle is central to the Earth's climate system, with rainfall and the physics of its formation acting as the key links in the cycle. Two-thirds of global rainfall occurs in the Tropics. Associated with this rainfall is a vast amount of heat, which is known as latent heat. It arises mainly due to the phase change of water vapor condensing into liquid droplets; three-fourths of the total heat energy available to the Earth's atmosphere comes from tropical rainfall. In addition, fresh water provided by tropical rainfall and its variability exerts a large impact upon the structure and motions of the upper ocean layer. An improved convective -stratiform heating (CSH) algorithm has been developed to obtain the 3D structure of cloud heating over the Tropics based on two sources of information: 1) rainfall information, namely its amount and the fraction due to light rain intensity, observed directly from the Precipitation Radar (PR) on board the TRMM satellite and 2) synthetic cloud physics information obtained from cloud-resolving model (CRM) simulations of cloud systems. The cloud simulations provide details on cloud processes, specifically latent heating, eddy heat flux convergence and radiative heating/cooling, that. are not directly observable by satellite. The new CSH algorithm-derived heating has a noticeably different heating structure over both ocean and land regions compared to the previous CSH algorithm. One of the major differences between new and old algorithms is that the level of maximum cloud heating occurs 1 to 1.5 km lower in the atmosphere in the new algorithm. This can effect the structure of the implied air currents associated with the general circulation of the atmosphere in the Tropics. The new CSH algorithm will be used provide retrieved heating data to other heating algorithms to supplement their performance.

Satellite Data Sets in the Polar Regions

NASA Technical Reports Server (NTRS)

Comiso, Josefino C.; Busalacchi, Antonio J. (Technical Monitor)

2000-01-01

We have generated about two decades of consistently derived geophysical parameters in the polar regions. The key parameters are sea ice concentration, surface temperature, albedo, and cloud cover statistics. Sea ice concentrations were derived from the Scanning Multichannel Microwave Radiometer (SMMR) data and the Special Scanning Cl Microwave Imager (SSM/I) data from several platforms using the enhanced Bootstrap Algorithm for the period 1978 through 1999. The new algorithm reduces the errors associated with spatial and temporal variations in the emissivity and surface temperatures of sea ice. Also, bad data at ocean/land interfaces are identified and deleted in an unsupervised manner. Surface ice temperature, albedo and cloud cover statistics are derived simultaneously from the Advanced Very High Resolution Radiometer (AVHRR) data from 1981 through 1999 and mapped at a higher resolution but the same format as the ice concentration data. The technique makes use these co-registered ice concentration maps to enable cloud masking to be done separately for open ocean, sea ice and land areas. The effect of inversion is minimized by taking into consideration the expected changes in the effect of inversion with altitude, especially in the Antarctic. A technique for ice type regional classification has also been developed using multichannel cluster analysis and a neural network. This provide a means to identify large areas of thin ice, first year ice, and older ice types. The data sets have been shown to be coherent with each other and provide a powerful tool for in depth studies of the currently changing Arctic and Antarctic environment.

Toward a 30m resolution time series of historical global urban expansion: exploring variation in North American cities

NASA Astrophysics Data System (ADS)

Stuhlmacher, M.; Wang, C.; Georgescu, M.; Tellman, B.; Balling, R.; Clinton, N. E.; Collins, L.; Goldblatt, R.; Hanson, G.

2016-12-01

Global representations of modern day urban land use and land cover (LULC) extent are becoming increasingly prevalent. Yet considerable uncertainties in the representation of built environment extent (i.e. global classifications generated from 250m resolution MODIS imagery or the United States' National Land Cover Database) remain because of the lack of a systematic, globally consistent methodological approach. We aim to increase resolution, accuracy, and improve upon past efforts by establishing a data-driven definition of the urban landscape, based on Landsat 5, 7 & 8 imagery and ancillary data sets. Continuous and discrete machine learning classification algorithms have been developed in Google Earth Engine (GEE), a powerful online cloud-based geospatial storage and parallel-computing platform. Additionally, thousands of ground truth points have been selected from high resolution imagery to fill in the previous lack of accurate data to be used for training and validation. We will present preliminary classification and accuracy assessments for select cities in the United States and Mexico. Our approach has direct implications for development of projected urban growth that is grounded on realistic identification of urbanizing hot-spots, with consequences for local to regional scale climate change, energy demand, water stress, human health, urban-ecological interactions, and efforts used to prioritize adaptation and mitigation strategies to offset large-scale climate change. Future work to apply the built-up detection algorithm globally and yearly is underway in a partnership between GEE, University of California in San Diego, and Arizona State University.

Retrieve Optically Thick Ice Cloud Microphysical Properties by Using Airborne Dual-Wavelength Radar Measurements

NASA Technical Reports Server (NTRS)

Wang, Zhien; Heymsfield, Gerald M.; Li, Lihua; Heymsfield, Andrew J.

2005-01-01

An algorithm to retrieve optically thick ice cloud microphysical property profiles is developed by using the GSFC 9.6 GHz ER-2 Doppler Radar (EDOP) and the 94 GHz Cloud Radar System (CRS) measurements aboard the high-altitude ER-2 aircraft. In situ size distribution and total water content data from the CRYSTAL-FACE field campaign are used for the algorithm development. To reduce uncertainty in calculated radar reflectivity factors (Ze) at these wavelengths, coincident radar measurements and size distribution data are used to guide the selection of mass-length relationships and to deal with the density and non-spherical effects of ice crystals on the Ze calculations. The algorithm is able to retrieve microphysical property profiles of optically thick ice clouds, such as, deep convective and anvil clouds, which are very challenging for single frequency radar and lidar. Examples of retrieved microphysical properties for a deep convective clouds are presented, which show that EDOP and CRS measurements provide rich information to study cloud structure and evolution. Good agreement between IWPs derived from an independent submillimeter-wave radiometer, CoSSIR, and dual-wavelength radar measurements indicates accuracy of the IWC retrieved from the two-frequency radar algorithm.

Equivalent Sensor Radiance Generation and Remote Sensing from Model Parameters. Part 1; Equivalent Sensor Radiance Formulation

NASA Technical Reports Server (NTRS)

Wind, Galina; DaSilva, Arlindo M.; Norris, Peter M.; Platnick, Steven E.

2013-01-01

In this paper we describe a general procedure for calculating equivalent sensor radiances from variables output from a global atmospheric forecast model. In order to take proper account of the discrepancies between model resolution and sensor footprint the algorithm takes explicit account of the model subgrid variability, in particular its description of the probably density function of total water (vapor and cloud condensate.) The equivalent sensor radiances are then substituted into an operational remote sensing algorithm processing chain to produce a variety of remote sensing products that would normally be produced from actual sensor output. This output can then be used for a wide variety of purposes such as model parameter verification, remote sensing algorithm validation, testing of new retrieval methods and future sensor studies. We show a specific implementation using the GEOS-5 model, the MODIS instrument and the MODIS Adaptive Processing System (MODAPS) Data Collection 5.1 operational remote sensing cloud algorithm processing chain (including the cloud mask, cloud top properties and cloud optical and microphysical properties products.) We focus on clouds and cloud/aerosol interactions, because they are very important to model development and improvement.

Multi-sensor Cloud Retrieval Simulator and Remote Sensing from Model Parameters . Pt. 1; Synthetic Sensor Radiance Formulation; [Synthetic Sensor Radiance Formulation

NASA Technical Reports Server (NTRS)

Wind, G.; DaSilva, A. M.; Norris, P. M.; Platnick, S.

2013-01-01

In this paper we describe a general procedure for calculating synthetic sensor radiances from variable output from a global atmospheric forecast model. In order to take proper account of the discrepancies between model resolution and sensor footprint, the algorithm takes explicit account of the model subgrid variability, in particular its description of the probability density function of total water (vapor and cloud condensate.) The simulated sensor radiances are then substituted into an operational remote sensing algorithm processing chain to produce a variety of remote sensing products that would normally be produced from actual sensor output. This output can then be used for a wide variety of purposes such as model parameter verification, remote sensing algorithm validation, testing of new retrieval methods and future sensor studies.We show a specific implementation using the GEOS-5 model, the MODIS instrument and the MODIS Adaptive Processing System (MODAPS) Data Collection 5.1 operational remote sensing cloud algorithm processing chain (including the cloud mask, cloud top properties and cloud optical and microphysical properties products). We focus on clouds because they are very important to model development and improvement.

A new task scheduling algorithm based on value and time for cloud platform

NASA Astrophysics Data System (ADS)

Kuang, Ling; Zhang, Lichen

2017-08-01

Tasks scheduling, a key part of increasing resource utilization and enhancing system performance, is a never outdated problem especially in cloud platforms. Based on the value density algorithm of the real-time task scheduling system and the character of the distributed system, the paper present a new task scheduling algorithm by further studying the cloud technology and the real-time system: Least Level Value Density First (LLVDF). The algorithm not only introduces some attributes of time and value for tasks, it also can describe weighting relationships between these properties mathematically. As this feature of the algorithm, it can gain some advantages to distinguish between different tasks more dynamically and more reasonably. When the scheme was used in the priority calculation of the dynamic task scheduling on cloud platform, relying on its advantage, it can schedule and distinguish tasks with large amounts and many kinds more efficiently. The paper designs some experiments, some distributed server simulation models based on M/M/C model of queuing theory and negative arrivals, to compare the algorithm against traditional algorithm to observe and show its characters and advantages.

Snow cover detection algorithm using dynamic time warping method and reflectances of MODIS solar spectrum channels

NASA Astrophysics Data System (ADS)

Lee, Kyeong-sang; Choi, Sungwon; Seo, Minji; Lee, Chang suk; Seong, Noh-hun; Han, Kyung-Soo

2016-10-01

Snow cover is biggest single component of cryosphere. The Snow is covering the ground in the Northern Hemisphere approximately 50% in winter season and is one of climate factors that affects Earth's energy budget because it has higher reflectance than other land types. Also, snow cover has an important role about hydrological modeling and water resource management. For this reason, accurate detection of snow cover acts as an essential element for regional water resource management. Snow cover detection using satellite-based data have some advantages such as obtaining wide spatial range data and time-series observations periodically. In the case of snow cover detection using satellite data, the discrimination of snow and cloud is very important. Typically, Misclassified cloud and snow pixel can lead directly to error factor for retrieval of satellite-based surface products. However, classification of snow and cloud is difficult because cloud and snow have similar optical characteristics and are composed of water or ice. But cloud and snow has different reflectance in 1.5 1.7 μm wavelength because cloud has lower grain size and moisture content than snow. So, cloud and snow shows difference reflectance patterns change according to wavelength. Therefore, in this study, we perform algorithm for classifying snow cover and cloud with satellite-based data using Dynamic Time Warping (DTW) method which is one of commonly used pattern analysis such as speech and fingerprint recognitions and reflectance spectral library of snow and cloud. Reflectance spectral library is constructed in advance using MOD21km (MODIS Level1 swath 1km) data that their reflectance is six channels including 3 (0.466μm), 4 (0.554μm), 1 (0.647μm), 2 (0.857μm), 26 (1.382μm) and 6 (1.629μm). We validate our result using MODIS RGB image and MOD10 L2 swath (MODIS swath snow cover product). And we use PA (Producer's Accuracy), UA (User's Accuracy) and CI (Comparison Index) as validation criteria. The result of our study detect as snow cover in the several regions which are did not detected as snow in MOD10 L2 and detected as snow cover in MODIS RGB image. The result of our study can improve accuracy of other surface product such as land surface reflectance and land surface emissivity. Also it can use input data of hydrological modeling.

Pan-sharpening algorithm to remove thin cloud via mask dodging and nonsampled shift-invariant shearlet transform

NASA Astrophysics Data System (ADS)

Shi, Cheng; Liu, Fang; Li, Ling-Ling; Hao, Hong-Xia

2014-01-01

The goal of pan-sharpening is to get an image with higher spatial resolution and better spectral information. However, the resolution of the pan-sharpened image is seriously affected by the thin clouds. For a single image, filtering algorithms are widely used to remove clouds. These kinds of methods can remove clouds effectively, but the detail lost in the cloud removal image is also serious. To solve this problem, a pan-sharpening algorithm to remove thin cloud via mask dodging and nonsampled shift-invariant shearlet transform (NSST) is proposed. For the low-resolution multispectral (LR MS) and high-resolution panchromatic images with thin clouds, a mask dodging method is used to remove clouds. For the cloud removal LR MS image, an adaptive principal component analysis transform is proposed to balance the spectral information and spatial resolution in the pan-sharpened image. Since the clouds removal process causes the detail loss problem, a weight matrix is designed to enhance the details of the cloud regions in the pan-sharpening process, but noncloud regions remain unchanged. And the details of the image are obtained by NSST. Experimental results over visible and evaluation metrics demonstrate that the proposed method can keep better spectral information and spatial resolution, especially for the images with thin clouds.

ASSURED CLOUD COMPUTING UNIVERSITY CENTER OFEXCELLENCE (ACC UCOE)

DTIC Science & Technology

2018-01-18

average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed...infrastructure security -Design of algorithms and techniques for real- time assuredness in cloud computing -Map-reduce task assignment with data locality...46 DESIGN OF ALGORITHMS AND TECHNIQUES FOR REAL- TIME ASSUREDNESS IN CLOUD COMPUTING

Cloud Properties and Radiative Heating Rates for TWP

DOE Data Explorer

Comstock, Jennifer

2013-11-07

A cloud properties and radiative heating rates dataset is presented where cloud properties retrieved using lidar and radar observations are input into a radiative transfer model to compute radiative fluxes and heating rates at three ARM sites located in the Tropical Western Pacific (TWP) region. The cloud properties retrieval is a conditional retrieval that applies various retrieval techniques depending on the available data, that is if lidar, radar or both instruments detect cloud. This Combined Remote Sensor Retrieval Algorithm (CombRet) produces vertical profiles of liquid or ice water content (LWC or IWC), droplet effective radius (re), ice crystal generalized effective size (Dge), cloud phase, and cloud boundaries. The algorithm was compared with 3 other independent algorithms to help estimate the uncertainty in the cloud properties, fluxes, and heating rates (Comstock et al. 2013). The dataset is provided at 2 min temporal and 90 m vertical resolution. The current dataset is applied to time periods when the MMCR (Millimeter Cloud Radar) version of the ARSCL (Active Remotely-Sensed Cloud Locations) Value Added Product (VAP) is available. The MERGESONDE VAP is utilized where temperature and humidity profiles are required. Future additions to this dataset will utilize the new KAZR instrument and its associated VAPs.

Failure and Redemption of Multifilter Rotating Shadowband Radiometer (MFRSR)/Normal Incidence Multifilter Radiometer (NIMFR) Cloud Screening: Contrasting Algorithm Performance at Atmospheric Radiation Measurement (ARM) North Slope of Alaska (NSA) and Southern Great Plains (SGP) Sites

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kassianov, Evgueni I.; Flynn, Connor J.; Koontz, Annette S.

2013-09-11

Well-known cloud-screening algorithms, which are designed to remove cloud-contaminated aerosol optical depths (AOD) from AOD measurements, have shown great performance at many middle-to-low latitude sites around the world. However, they may occasionally fail under challenging observational conditions, such as when the sun is low (near the horizon) or when optically thin clouds with small spatial inhomogeneity occur. Such conditions have been observed quite frequently at the high-latitude Atmospheric Radiation Measurement (ARM) North Slope of Alaska (NSA) sites. A slightly modified cloud-screening version of the standard algorithm is proposed here with a focus on the ARM-supported Multifilter Rotating Shadowband Radiometer (MFRSR)more » and Normal Incidence Multifilter Radiometer (NIMFR) data. The modified version uses approximately the same techniques as the standard algorithm, but it additionally examines the magnitude of the slant-path line of sight transmittance and eliminates points when the observed magnitude is below a specified threshold. Substantial improvement of the multi-year (1999-2012) aerosol product (AOD and its Angstrom exponent) is shown for the NSA sites when the modified version is applied. Moreover, this version reproduces the AOD product at the ARM Southern Great Plains (SGP) site, which was originally generated by the standard cloud-screening algorithms. The proposed minor modification is easy to implement and its application to existing and future cloud-screening algorithms can be particularly beneficial for challenging observational conditions.« less

A cloud and radiation model-based algorithm for rainfall retrieval from SSM/I multispectral microwave measurements

NASA Technical Reports Server (NTRS)

Xiang, Xuwu; Smith, Eric A.; Tripoli, Gregory J.

1992-01-01

A hybrid statistical-physical retrieval scheme is explored which combines a statistical approach with an approach based on the development of cloud-radiation models designed to simulate precipitating atmospheres. The algorithm employs the detailed microphysical information from a cloud model as input to a radiative transfer model which generates a cloud-radiation model database. Statistical procedures are then invoked to objectively generate an initial guess composite profile data set from the database. The retrieval algorithm has been tested for a tropical typhoon case using Special Sensor Microwave/Imager (SSM/I) data and has shown satisfactory results.

Cloud cover and solar disk state estimation using all-sky images: deep neural networks approach compared to routine methods

NASA Astrophysics Data System (ADS)

Krinitskiy, Mikhail; Sinitsyn, Alexey

2017-04-01

Shortwave radiation is an important component of surface heat budget over sea and land. To estimate them accurate observations of cloud conditions are needed including total cloud cover, spatial and temporal cloud structure. While massively observed visually, for building accurate SW radiation parameterizations cloud structure needs also to be quantified using precise instrumental measurements. While there already exist several state of the art land-based cloud-cameras that satisfy researchers needs, their major disadvantages are associated with inaccuracy of all-sky images processing algorithms which typically result in the uncertainties of 2-4 octa of cloud cover estimates with the resulting true-scoring cloud cover accuracy of about 7%. Moreover, none of these algorithms determine cloud types. We developed an approach for cloud cover and structure estimating, which provides much more accurate estimates and also allows for measuring additional characteristics. This method is based on the synthetic controlling index, namely the "grayness rate index", that we introduced in 2014. Since then this index has already demonstrated high efficiency being used along with the technique namely the "background sunburn effect suppression", to detect thin clouds. This made it possible to significantly increase the accuracy of total cloud cover estimation in various sky image states using this extension of routine algorithm type. Errors for the cloud cover estimates significantly decreased down resulting the mean squared error of about 1.5 octa. Resulting true-scoring accuracy is more than 38%. The main source of this approach uncertainties is the solar disk state determination errors. While the deep neural networks approach lets us to estimate solar disk state with 94% accuracy, the final result of total cloud estimation still isn`t satisfying. To solve this problem completely we applied the set of machine learning algorithms to the problem of total cloud cover estimation directly. The accuracy of this approach varies depending on algorithm choice. Deep neural networks demonstrated the best accuracy of more than 96%. We will demonstrate some approaches and the most influential statistical features of all-sky images that lets the algorithm reach that high accuracy. With the use of our new optical package a set of over 480`000 samples has been collected in several sea missions in 2014-2016 along with concurrent standard human observed and instrumentally recorded meteorological parameters. We will demonstrate the results of the field measurements and will discuss some still remaining problems and the potential of the further developments of machine learning approach.

Satellite Imagery Analysis for Automated Global Food Security Forecasting

NASA Astrophysics Data System (ADS)

Moody, D.; Brumby, S. P.; Chartrand, R.; Keisler, R.; Mathis, M.; Beneke, C. M.; Nicholaeff, D.; Skillman, S.; Warren, M. S.; Poehnelt, J.

2017-12-01

The recent computing performance revolution has driven improvements in sensor, communication, and storage technology. Multi-decadal remote sensing datasets at the petabyte scale are now available in commercial clouds, with new satellite constellations generating petabytes/year of daily high-resolution global coverage imagery. Cloud computing and storage, combined with recent advances in machine learning, are enabling understanding of the world at a scale and at a level of detail never before feasible. We present results from an ongoing effort to develop satellite imagery analysis tools that aggregate temporal, spatial, and spectral information and that can scale with the high-rate and dimensionality of imagery being collected. We focus on the problem of monitoring food crop productivity across the Middle East and North Africa, and show how an analysis-ready, multi-sensor data platform enables quick prototyping of satellite imagery analysis algorithms, from land use/land cover classification and natural resource mapping, to yearly and monthly vegetative health change trends at the structural field level.

Conceptual design of the CZMIL data processing system (DPS): algorithms and software for fusing lidar, hyperspectral data, and digital images

NASA Astrophysics Data System (ADS)

Park, Joong Yong; Tuell, Grady

2010-04-01

The Data Processing System (DPS) of the Coastal Zone Mapping and Imaging Lidar (CZMIL) has been designed to automatically produce a number of novel environmental products through the fusion of Lidar, spectrometer, and camera data in a single software package. These new products significantly transcend use of the system as a bathymeter, and support use of CZMIL as a complete coastal and benthic mapping tool. The DPS provides a spinning globe capability for accessing data files; automated generation of combined topographic and bathymetric point clouds; a fully-integrated manual editor and data analysis tool; automated generation of orthophoto mosaics; automated generation of reflectance data cubes from the imaging spectrometer; a coupled air-ocean spectral optimization model producing images of chlorophyll and CDOM concentrations; and a fusion based capability to produce images and classifications of the shallow water seafloor. Adopting a multitasking approach, we expect to achieve computation of the point clouds, DEMs, and reflectance images at a 1:1 processing to acquisition ratio.

Content-based histopathology image retrieval using CometCloud.

PubMed

Qi, Xin; Wang, Daihou; Rodero, Ivan; Diaz-Montes, Javier; Gensure, Rebekah H; Xing, Fuyong; Zhong, Hua; Goodell, Lauri; Parashar, Manish; Foran, David J; Yang, Lin

2014-08-26

The development of digital imaging technology is creating extraordinary levels of accuracy that provide support for improved reliability in different aspects of the image analysis, such as content-based image retrieval, image segmentation, and classification. This has dramatically increased the volume and rate at which data are generated. Together these facts make querying and sharing non-trivial and render centralized solutions unfeasible. Moreover, in many cases this data is often distributed and must be shared across multiple institutions requiring decentralized solutions. In this context, a new generation of data/information driven applications must be developed to take advantage of the national advanced cyber-infrastructure (ACI) which enable investigators to seamlessly and securely interact with information/data which is distributed across geographically disparate resources. This paper presents the development and evaluation of a novel content-based image retrieval (CBIR) framework. The methods were tested extensively using both peripheral blood smears and renal glomeruli specimens. The datasets and performance were evaluated by two pathologists to determine the concordance. The CBIR algorithms that were developed can reliably retrieve the candidate image patches exhibiting intensity and morphological characteristics that are most similar to a given query image. The methods described in this paper are able to reliably discriminate among subtle staining differences and spatial pattern distributions. By integrating a newly developed dual-similarity relevance feedback module into the CBIR framework, the CBIR results were improved substantially. By aggregating the computational power of high performance computing (HPC) and cloud resources, we demonstrated that the method can be successfully executed in minutes on the Cloud compared to weeks using standard computers. In this paper, we present a set of newly developed CBIR algorithms and validate them using two different pathology applications, which are regularly evaluated in the practice of pathology. Comparative experimental results demonstrate excellent performance throughout the course of a set of systematic studies. Additionally, we present and evaluate a framework to enable the execution of these algorithms across distributed resources. We show how parallel searching of content-wise similar images in the dataset significantly reduces the overall computational time to ensure the practical utility of the proposed CBIR algorithms.

Quantifying the Climate-Scale Accuracy of Satellite Cloud Retrievals

NASA Astrophysics Data System (ADS)

Roberts, Y.; Wielicki, B. A.; Sun-Mack, S.; Minnis, P.; Liang, L.; Di Girolamo, L.

2014-12-01

Instrument calibration and cloud retrieval algorithms have been developed to minimize retrieval errors on small scales. However, measurement uncertainties and assumptions within retrieval algorithms at the pixel level may alias into decadal-scale trends of cloud properties. We first, therefore, quantify how instrument calibration changes could alias into cloud property trends. For a perfect observing system the climate trend accuracy is limited only by the natural variability of the climate variable. Alternatively, for an actual observing system, the climate trend accuracy is additionally limited by the measurement uncertainty. Drifts in calibration over time may therefore be disguised as a true climate trend. We impose absolute calibration changes to MODIS spectral reflectance used as input to the CERES Cloud Property Retrieval System (CPRS) and run the modified MODIS reflectance through the CPRS to determine the sensitivity of cloud properties to calibration changes. We then use these changes to determine the impact of instrument calibration changes on trend uncertainty in reflected solar cloud properties. Secondly, we quantify how much cloud retrieval algorithm assumptions alias into cloud optical retrieval trends by starting with the largest of these biases: the plane-parallel assumption in cloud optical thickness (τC) retrievals. First, we collect liquid water cloud fields obtained from Multi-angle Imaging Spectroradiometer (MISR) measurements to construct realistic probability distribution functions (PDFs) of 3D cloud anisotropy (a measure of the degree to which clouds depart from plane-parallel) for different ISCCP cloud types. Next, we will conduct a theoretical study with dynamically simulated cloud fields and a 3D radiative transfer model to determine the relationship between 3D cloud anisotropy and 3D τC bias for each cloud type. Combining these results provides distributions of 3D τC bias by cloud type. Finally, we will estimate the change in frequency of occurrence of cloud types between two decades and will have the information needed to calculate the total change in 3D optical thickness bias between two decades. If we uncover aliases in this study, the results will motivate the development and rigorous testing of climate specific cloud retrieval algorithms.

A macrophysical life cycle description for precipitating systems

NASA Astrophysics Data System (ADS)

Evaristo, Raquel; Xie, Xinxin; Troemel, Silke; Diederich, Malte; Simon, Juergen; Simmer, Clemens

2014-05-01

The lack of understanding of cloud and precipitation processes is still the overarching problem of climate simulation, and prediction. The work presented is part of the HD(CP)2 project (High Definition Clouds and Precipitation for Advancing Climate Predictions) which aims at building a very high resolution model in order to evaluate and exploit regional hindcasts for the purpose of parameterization development. To this end, an observational object-based climatology for precipitation systems will be built, and shall later be compared with a twin model-based climatological data base for pseudo precipitation events within an event-based model validation approach. This is done by identifying internal structures, described by means of macrophysical descriptors used to characterize the temporal development of tracked rain events. 2 pre-requisites are necessary for this: 1) a tracking algorithm, and 2) 3D radar/satellite composite. Both prerequisites are ready to be used, and have already been applied to a few case studies. Some examples of these macrophysical descriptors are differential reflectivity columns, bright band fraction and trend, cloud top heights, the spatial extent of updrafts or downdrafts or the ice content. We will show one case study from August 5th 2012, when convective precipitation was observed simultaneously by the BOXPOL and JUXPOL X-band polarimetric radars. We will follow the main paths identified by the tracking algorithm during this event and identify in the 3D composite the descriptors that characterize precipitation development, their temporal evolution, and the different macrophysical processes that are ultimately related to the precipitation observed. In a later stage these observations will be compared to the results of hydrometeor classification algorithm, in order to link the macrophysical and microphysical aspects of the storm evolution. The detailed microphysical processes are the subject of a closely related work also presented in this session: Microphysical processes observed by X band polarimetric radars during the evolution of storm systems, by Xinxin Xie et al.

A Novel Deployment Method for Communication-Intensive Applications in Service Clouds

PubMed Central

Liu, Chuanchang; Yang, Jingqi

2014-01-01

The service platforms are migrating to clouds for reasonably solving long construction periods, low resource utilizations, and isolated constructions of service platforms. However, when the migration is conducted in service clouds, there is a little focus of deploying communication-intensive applications in previous deployment methods. To address this problem, this paper proposed the combination of the online deployment and the offline deployment for deploying communication-intensive applications in service clouds. Firstly, the system architecture was designed for implementing the communication-aware deployment method for communication-intensive applications in service clouds. Secondly, in the online-deployment algorithm and the offline-deployment algorithm, service instances were deployed in an optimal cloud node based on the communication overhead which is determined by the communication traffic between services, as well as the communication performance between cloud nodes. Finally, the experimental results demonstrated that the proposed methods deployed communication-intensive applications effectively with lower latency and lower load compared with existing algorithms. PMID:25140331

Cloud Retrieval Intercomparisons Between SEVIRI, MODIS and VIIRS with CHIMAERA PGE06 Data Collection 6 Products

NASA Technical Reports Server (NTRS)

Wind, Galina; Riedi, Jerome; Platnick, Steven; Heidinger, Andrew

2014-01-01

The Cross-platform HIgh resolution Multi-instrument AtmosphEric Retrieval Algorithms (CHIMAERA) system allows us to perform MODIS-like cloud top, optical and microphysical properties retrievals on any sensor that possesses a minimum set of common spectral channels. The CHIMAERA system uses a shared-core architecture that takes retrieval method out of the equation when intercomparisons are made. Here we show an example of such retrieval and a comparison of simultaneous retrievals done using SEVIRI, MODIS and VIIRS sensors. All sensor retrievals are performed using CLAVR-x (or CLAVR-x based) cloud top properties algorithm. SEVIRI uses the SAF_NWC cloud mask. MODIS and VIIRS use the IFF-based cloud mask that is a shared algorithm between MODIS and VIIRS. The MODIS and VIIRS retrievals are performed using a VIIRS branch of CHIMAERA that limits available MODIS channel set. Even though in that mode certain MODIS products such as multilayer cloud map are not available, the cloud retrieval remains fully equivalent to operational Data Collection 6.

A novel deployment method for communication-intensive applications in service clouds.

PubMed

Liu, Chuanchang; Yang, Jingqi

2014-01-01

The service platforms are migrating to clouds for reasonably solving long construction periods, low resource utilizations, and isolated constructions of service platforms. However, when the migration is conducted in service clouds, there is a little focus of deploying communication-intensive applications in previous deployment methods. To address this problem, this paper proposed the combination of the online deployment and the offline deployment for deploying communication-intensive applications in service clouds. Firstly, the system architecture was designed for implementing the communication-aware deployment method for communication-intensive applications in service clouds. Secondly, in the online-deployment algorithm and the offline-deployment algorithm, service instances were deployed in an optimal cloud node based on the communication overhead which is determined by the communication traffic between services, as well as the communication performance between cloud nodes. Finally, the experimental results demonstrated that the proposed methods deployed communication-intensive applications effectively with lower latency and lower load compared with existing algorithms.

OCRA radiometric cloud fractions for GOME-2 on MetOp-A/B

NASA Astrophysics Data System (ADS)

Lutz, R.; Loyola, D.; Gimeno García, S.; Romahn, F.

2015-12-01

This paper describes an approach for cloud parameter retrieval (radiometric cloud fraction estimation) using the polarization measurements of the Global Ozone Monitoring Experiment-2 (GOME-2) on-board the MetOp-A/B satellites. The core component of the Optical Cloud Recognition Algorithm (OCRA) is the calculation of monthly cloud-free reflectances for a global grid (resolution of 0.2° in longitude and 0.2° in latitude) and to derive radiometric cloud fractions. These cloud fractions will serve as a priori information for the retrieval of cloud top height (CTH), cloud top pressure (CTP), cloud top albedo (CTA) and cloud optical thickness (COT) with the Retrieval Of Cloud Information using Neural Networks (ROCINN) algorithm. This approach is already being implemented operationally for the GOME/ERS-2 and SCIAMACHY/ENVISAT sensors and here we present version 3.0 of the OCRA algorithm applied to the GOME-2 sensors. Based on more than six years of GOME-2A data (February 2007-June 2013), reflectances are calculated for ≈ 35 000 orbits. For each measurement a degradation correction as well as a viewing angle dependent and latitude dependent correction is applied. In addition, an empirical correction scheme is introduced in order to remove the effect of oceanic sun glint. A comparison of the GOME-2A/B OCRA cloud fractions with co-located AVHRR geometrical cloud fractions shows a general good agreement with a mean difference of -0.15±0.20. From operational point of view, an advantage of the OCRA algorithm is its extremely fast computational time and its straightforward transferability to similar sensors like OMI (Ozone Monitoring Instrument), TROPOMI (TROPOspheric Monitoring Instrument) on Sentinel 5 Precursor, as well as Sentinel 4 and Sentinel 5. In conclusion, it is shown that a robust, accurate and fast radiometric cloud fraction estimation for GOME-2 can be achieved with OCRA by using the polarization measurement devices (PMDs).

OCRA radiometric cloud fractions for GOME-2 on MetOp-A/B

NASA Astrophysics Data System (ADS)

Lutz, Ronny; Loyola, Diego; Gimeno García, Sebastián; Romahn, Fabian

2016-05-01

This paper describes an approach for cloud parameter retrieval (radiometric cloud-fraction estimation) using the polarization measurements of the Global Ozone Monitoring Experiment-2 (GOME-2) onboard the MetOp-A/B satellites. The core component of the Optical Cloud Recognition Algorithm (OCRA) is the calculation of monthly cloud-free reflectances for a global grid (resolution of 0.2° in longitude and 0.2° in latitude) to derive radiometric cloud fractions. These cloud fractions will serve as a priori information for the retrieval of cloud-top height (CTH), cloud-top pressure (CTP), cloud-top albedo (CTA) and cloud optical thickness (COT) with the Retrieval Of Cloud Information using Neural Networks (ROCINN) algorithm. This approach is already being implemented operationally for the GOME/ERS-2 and SCIAMACHY/ENVISAT sensors and here we present version 3.0 of the OCRA algorithm applied to the GOME-2 sensors. Based on more than five years of GOME-2A data (April 2008 to June 2013), reflectances are calculated for ≈ 35 000 orbits. For each measurement a degradation correction as well as a viewing-angle-dependent and latitude-dependent correction is applied. In addition, an empirical correction scheme is introduced in order to remove the effect of oceanic sun glint. A comparison of the GOME-2A/B OCRA cloud fractions with colocated AVHRR (Advanced Very High Resolution Radiometer) geometrical cloud fractions shows a general good agreement with a mean difference of -0.15 ± 0.20. From an operational point of view, an advantage of the OCRA algorithm is its very fast computational time and its straightforward transferability to similar sensors like OMI (Ozone Monitoring Instrument), TROPOMI (TROPOspheric Monitoring Instrument) on Sentinel 5 Precursor, as well as Sentinel 4 and Sentinel 5. In conclusion, it is shown that a robust, accurate and fast radiometric cloud-fraction estimation for GOME-2 can be achieved with OCRA using polarization measurement devices (PMDs).

Generic-distributed framework for cloud services marketplace based on unified ontology.

PubMed

Hasan, Samer; Valli Kumari, V

2017-11-01

Cloud computing is a pattern for delivering ubiquitous and on demand computing resources based on pay-as-you-use financial model. Typically, cloud providers advertise cloud service descriptions in various formats on the Internet. On the other hand, cloud consumers use available search engines (Google and Yahoo) to explore cloud service descriptions and find the adequate service. Unfortunately, general purpose search engines are not designed to provide a small and complete set of results, which makes the process a big challenge. This paper presents a generic-distrusted framework for cloud services marketplace to automate cloud services discovery and selection process, and remove the barriers between service providers and consumers. Additionally, this work implements two instances of generic framework by adopting two different matching algorithms; namely dominant and recessive attributes algorithm borrowed from gene science and semantic similarity algorithm based on unified cloud service ontology. Finally, this paper presents unified cloud services ontology and models the real-life cloud services according to the proposed ontology. To the best of the authors' knowledge, this is the first attempt to build a cloud services marketplace where cloud providers and cloud consumers can trend cloud services as utilities. In comparison with existing work, semantic approach reduced the execution time by 20% and maintained the same values for all other parameters. On the other hand, dominant and recessive attributes approach reduced the execution time by 57% but showed lower value for recall.

a Threshold-Free Filtering Algorithm for Airborne LIDAR Point Clouds Based on Expectation-Maximization

NASA Astrophysics Data System (ADS)

Hui, Z.; Cheng, P.; Ziggah, Y. Y.; Nie, Y.

2018-04-01

Filtering is a key step for most applications of airborne LiDAR point clouds. Although lots of filtering algorithms have been put forward in recent years, most of them suffer from parameters setting or thresholds adjusting, which will be time-consuming and reduce the degree of automation of the algorithm. To overcome this problem, this paper proposed a threshold-free filtering algorithm based on expectation-maximization. The proposed algorithm is developed based on an assumption that point clouds are seen as a mixture of Gaussian models. The separation of ground points and non-ground points from point clouds can be replaced as a separation of a mixed Gaussian model. Expectation-maximization (EM) is applied for realizing the separation. EM is used to calculate maximum likelihood estimates of the mixture parameters. Using the estimated parameters, the likelihoods of each point belonging to ground or object can be computed. After several iterations, point clouds can be labelled as the component with a larger likelihood. Furthermore, intensity information was also utilized to optimize the filtering results acquired using the EM method. The proposed algorithm was tested using two different datasets used in practice. Experimental results showed that the proposed method can filter non-ground points effectively. To quantitatively evaluate the proposed method, this paper adopted the dataset provided by the ISPRS for the test. The proposed algorithm can obtain a 4.48 % total error which is much lower than most of the eight classical filtering algorithms reported by the ISPRS.

Final Technical Report

DOE Office of Scientific and Technical Information (OSTI.GOV)

de Szoeke, Simon P.

The investigator and DOE-supported student [1] retrieved vertical air velocity and microphysical fall velocity retrieval for VOCALS and CAP-MBL homogeneous clouds. [2] Calculated in-cloud and cloud top dissipation calculation and diurnal cycle computed for VOCALS. [3] Compared CAP-MBL Doppler cloud radar scenes with (Remillard et al. 2012) automated classification.

A review of classification algorithms for EEG-based brain-computer interfaces.

PubMed

Lotte, F; Congedo, M; Lécuyer, A; Lamarche, F; Arnaldi, B

2007-06-01

In this paper we review classification algorithms used to design brain-computer interface (BCI) systems based on electroencephalography (EEG). We briefly present the commonly employed algorithms and describe their critical properties. Based on the literature, we compare them in terms of performance and provide guidelines to choose the suitable classification algorithm(s) for a specific BCI.

Automated Method of Frequency Determination in Software Metric Data Through the Use of the Multiple Signal Classification (MUSIC) Algorithm

DTIC Science & Technology

1998-06-26

METHOD OF FREQUENCY DETERMINATION 4 IN SOFTWARE METRIC DATA THROUGH THE USE OF THE 5 MULTIPLE SIGNAL CLASSIFICATION ( MUSIC ) ALGORITHM 6 7 STATEMENT OF...graph showing the estimated power spectral 12 density (PSD) generated by the multiple signal classification 13 ( MUSIC ) algorithm from the data set used...implemented in this module; however, it is preferred to use 1 the Multiple Signal Classification ( MUSIC ) algorithm. The MUSIC 2 algorithm is

Cloud Screening and Quality Control Algorithm for Star Photometer Data: Assessment with Lidar Measurements and with All-sky Images

NASA Technical Reports Server (NTRS)

Ramirez, Daniel Perez; Lyamani, H.; Olmo, F. J.; Whiteman, D. N.; Navas-Guzman, F.; Alados-Arboledas, L.

2012-01-01

This paper presents the development and set up of a cloud screening and data quality control algorithm for a star photometer based on CCD camera as detector. These algorithms are necessary for passive remote sensing techniques to retrieve the columnar aerosol optical depth, delta Ae(lambda), and precipitable water vapor content, W, at nighttime. This cloud screening procedure consists of calculating moving averages of delta Ae() and W under different time-windows combined with a procedure for detecting outliers. Additionally, to avoid undesirable Ae(lambda) and W fluctuations caused by the atmospheric turbulence, the data are averaged on 30 min. The algorithm is applied to the star photometer deployed in the city of Granada (37.16 N, 3.60 W, 680 ma.s.l.; South-East of Spain) for the measurements acquired between March 2007 and September 2009. The algorithm is evaluated with correlative measurements registered by a lidar system and also with all-sky images obtained at the sunset and sunrise of the previous and following days. Promising results are obtained detecting cloud-affected data. Additionally, the cloud screening algorithm has been evaluated under different aerosol conditions including Saharan dust intrusion, biomass burning and pollution events.

Improved wavelet packet classification algorithm for vibrational intrusions in distributed fiber-optic monitoring systems

NASA Astrophysics Data System (ADS)

Wang, Bingjie; Pi, Shaohua; Sun, Qi; Jia, Bo

2015-05-01

An improved classification algorithm that considers multiscale wavelet packet Shannon entropy is proposed. Decomposition coefficients at all levels are obtained to build the initial Shannon entropy feature vector. After subtracting the Shannon entropy map of the background signal, components of the strongest discriminating power in the initial feature vector are picked out to rebuild the Shannon entropy feature vector, which is transferred to radial basis function (RBF) neural network for classification. Four types of man-made vibrational intrusion signals are recorded based on a modified Sagnac interferometer. The performance of the improved classification algorithm has been evaluated by the classification experiments via RBF neural network under different diffusion coefficients. An 85% classification accuracy rate is achieved, which is higher than the other common algorithms. The classification results show that this improved classification algorithm can be used to classify vibrational intrusion signals in an automatic real-time monitoring system.

Merging Sounder and Imager Data for Improved Cloud Depiction on SNPP and JPSS.

NASA Astrophysics Data System (ADS)

Heidinger, A. K.; Holz, R.; Li, Y.; Platnick, S. E.; Wanzong, S.

2017-12-01

Under the NOAA GOES-R Algorithm Working Group (AWG) Program, NOAA supports the development of an Infrared (IR) Optimal Estimation (OE) Cloud Height Algorithm (ACHA). ACHA is an enterprise solution that supports many geostationary and polar orbiting imager sensors. ACHA is operational at NOAA on SNPP VIIRS and has been adopted as the cloud height algorithm for the NASA NPP Atmospheric Suite of products. Being an OE algorithm, ACHA is flexible and capable of using additional observations and constraints. We have modified ACHA to use sounder (CriS) observations to improve the cloud detection, typing and height estimation. Specifically, these improvements include retrievals in multi-layer scenarios and improved performance in polar regions. This presentation will describe the process for merging VIIRS and CrIS and a demonstration of the improvements.

Cloud Computing Security Model with Combination of Data Encryption Standard Algorithm (DES) and Least Significant Bit (LSB)

NASA Astrophysics Data System (ADS)

Basri, M.; Mawengkang, H.; Zamzami, E. M.

2018-03-01

Limitations of storage sources is one option to switch to cloud storage. Confidentiality and security of data stored on the cloud is very important. To keep up the confidentiality and security of such data can be done one of them by using cryptography techniques. Data Encryption Standard (DES) is one of the block cipher algorithms used as standard symmetric encryption algorithm. This DES will produce 8 blocks of ciphers combined into one ciphertext, but the ciphertext are weak against brute force attacks. Therefore, the last 8 block cipher will be converted into 8 random images using Least Significant Bit (LSB) algorithm which later draws the result of cipher of DES algorithm to be merged into one.

Three Way Comparison between Two OMI/Aura and One POLDER/PARASOL Cloud Pressure Products

NASA Technical Reports Server (NTRS)

Sneep, M.; deHaan, J. F.; Stammes, P.; Vanbaunce, C.; Joiner, J.; Vasilkov, A. P.; Levelt, P. F.

2007-01-01

The cloud pressures determined by three different algorithms, operating on reflectances measured by two space-borne instruments in the "A" train, are compared with each other. The retrieval algorithms are based on absorption in the oxygen A-band near 760 nm, absorption by a collision induced absorption in oxygen near 477nm, and the filling in of Fraunhofer lines by rotational Raman scattering. The first algorithm operates on data collected by the POLDER instrument on board PARASOL, while the latter two operate on data from the OMI instrument on board Aura. The satellites sample the same air mass within about 15 minutes. Using one month of data, the cloud pressures from the three algorithms are found to show a similar behavior, with correlation coefficients larger than 0.85 between the data sets for thick clouds. The average differences in the cloud pressure are also small, between 2 and 45 hPa, for the whole data set. For optically thin to medium thick clouds, the cloud pressure the distribution found by POLDER is very similar to that found by OMI using the O2 - O2 absorption. Somewhat larger differences are found for very thick clouds, and we hypothesise that the strong absorption in the oxygen A-band causes the POLDER instrument to retrieve lower pressures for those scenes.

Registration algorithm of point clouds based on multiscale normal features

NASA Astrophysics Data System (ADS)

Lu, Jun; Peng, Zhongtao; Su, Hang; Xia, GuiHua

2015-01-01

The point cloud registration technology for obtaining a three-dimensional digital model is widely applied in many areas. To improve the accuracy and speed of point cloud registration, a registration method based on multiscale normal vectors is proposed. The proposed registration method mainly includes three parts: the selection of key points, the calculation of feature descriptors, and the determining and optimization of correspondences. First, key points are selected from the point cloud based on the changes of magnitude of multiscale curvatures obtained by using principal components analysis. Then the feature descriptor of each key point is proposed, which consists of 21 elements based on multiscale normal vectors and curvatures. The correspondences in a pair of two point clouds are determined according to the descriptor's similarity of key points in the source point cloud and target point cloud. Correspondences are optimized by using a random sampling consistency algorithm and clustering technology. Finally, singular value decomposition is applied to optimized correspondences so that the rigid transformation matrix between two point clouds is obtained. Experimental results show that the proposed point cloud registration algorithm has a faster calculation speed, higher registration accuracy, and better antinoise performance.

Estimating vertical profiles of water-cloud droplet effective radius from SWIR satellite measurements via a statistical model derived from CloudSat observations

NASA Astrophysics Data System (ADS)

Nagao, T. M.; Murakami, H.; Nakajima, T. Y.

2017-12-01

This study proposes an algorithm to estimate vertical profiles of cloud droplet effective radius (CDER-VP) for water clouds from shortwave infrared (SWIR) measurements of Himawari-8/AHI via a statistical model of CDER-VP derived from CloudSat observation. Several similar algorithms in previous studies utilize a spectral radiance matching on the assumption of simultaneous observations of CloudSat and Aqua/MODIS. However, our algorithm does not assume simultaneous observations with CloudSat. First, in advance, a database (DB) of CDER-VP is prepared by the following procedure: TOA radiances at 0.65, 2.3 and 10.4-μm bands of the AHI are simulated using CDER-VP and cloud optical depth vertical profile (COD-VP) contained in the CloudSat 2B-CWC-RVOD and 2B-TAU products. Cloud optical thickness (COT), Column-CDER and cloud top height (CTH) are retrieved from the simulated radiances using a traditional retrieval algorithm with vertically homogeneous cloud model (1-SWIR VHC method). The CDER-VP is added to the DB by using the COT and Column-CDER retrievals as a key of the DB. Then by using principal component (PC) analysis, up to three PC vectors of the CDER-VPs in the DB are extracted. Next, the algorithm retrieves CDER-VP from actual AHI measurements by the following procedure: First, COT, Column-CDER and CTH are retrieved from TOA radiances at 0.65, 2.3 and 10.4-μm bands of the AHI using by 1-SWIR VHC method. Then, the PC vectors of CDER-VP is fetched from the DB using the COT and Column-CDER retrievals as the key of the DB. Finally, using coefficients of the PC vectors of CDER-VP as variables for retrieval, CDER-VP, COT and CTH are retrieved from TOA radiances at 0.65, 1.6, 2.3, 3.9 and 10.4-μm bands of the AHI based on optimal estimation method with iterative radiative transfer calculation. The simulation result showed the CDER-VP retrieval errors were almost smaller than 3 - 4 μm. The CDER retrieval errors at the cloud base were almost larger than the others (e.g. CDER at cloud top), especially when COT and CDER was large. The tendency can be explained by less sensitivities of SWIRs to CDER at cloud base. Additionally, as a case study, this study will attempt to apply the algorithm to the AHI's high-frequency observations, and to interpret the time series of the CDER-VP retrievals in terms of temporal evolution of water clouds.

Cloud based intelligent system for delivering health care as a service.

PubMed

Kaur, Pankaj Deep; Chana, Inderveer

2014-01-01

The promising potential of cloud computing and its convergence with technologies such as mobile computing, wireless networks, sensor technologies allows for creation and delivery of newer type of cloud services. In this paper, we advocate the use of cloud computing for the creation and management of cloud based health care services. As a representative case study, we design a Cloud Based Intelligent Health Care Service (CBIHCS) that performs real time monitoring of user health data for diagnosis of chronic illness such as diabetes. Advance body sensor components are utilized to gather user specific health data and store in cloud based storage repositories for subsequent analysis and classification. In addition, infrastructure level mechanisms are proposed to provide dynamic resource elasticity for CBIHCS. Experimental results demonstrate that classification accuracy of 92.59% is achieved with our prototype system and the predicted patterns of CPU usage offer better opportunities for adaptive resource elasticity. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

Comparative study of classification algorithms for damage classification in smart composite laminates

NASA Astrophysics Data System (ADS)

Khan, Asif; Ryoo, Chang-Kyung; Kim, Heung Soo

2017-04-01

This paper presents a comparative study of different classification algorithms for the classification of various types of inter-ply delaminations in smart composite laminates. Improved layerwise theory is used to model delamination at different interfaces along the thickness and longitudinal directions of the smart composite laminate. The input-output data obtained through surface bonded piezoelectric sensor and actuator is analyzed by the system identification algorithm to get the system parameters. The identified parameters for the healthy and delaminated structure are supplied as input data to the classification algorithms. The classification algorithms considered in this study are ZeroR, Classification via regression, Naïve Bayes, Multilayer Perceptron, Sequential Minimal Optimization, Multiclass-Classifier, and Decision tree (J48). The open source software of Waikato Environment for Knowledge Analysis (WEKA) is used to evaluate the classification performance of the classifiers mentioned above via 75-25 holdout and leave-one-sample-out cross-validation regarding classification accuracy, precision, recall, kappa statistic and ROC Area.

Tools for Analyzing Computing Resource Management Strategies and Algorithms for SDR Clouds

NASA Astrophysics Data System (ADS)

Marojevic, Vuk; Gomez-Miguelez, Ismael; Gelonch, Antoni

2012-09-01

Software defined radio (SDR) clouds centralize the computing resources of base stations. The computing resource pool is shared between radio operators and dynamically loads and unloads digital signal processing chains for providing wireless communications services on demand. Each new user session request particularly requires the allocation of computing resources for executing the corresponding SDR transceivers. The huge amount of computing resources of SDR cloud data centers and the numerous session requests at certain hours of a day require an efficient computing resource management. We propose a hierarchical approach, where the data center is divided in clusters that are managed in a distributed way. This paper presents a set of computing resource management tools for analyzing computing resource management strategies and algorithms for SDR clouds. We use the tools for evaluating a different strategies and algorithms. The results show that more sophisticated algorithms can achieve higher resource occupations and that a tradeoff exists between cluster size and algorithm complexity.

The Algorithm Theoretical Basis Document for the GLAS Atmospheric Data Products

NASA Technical Reports Server (NTRS)

Palm, Stephen P.; Hart, William D.; Hlavka, Dennis L.; Welton, Ellsworth J.; Spinhirne, James D.

2012-01-01

The purpose of this document is to present a detailed description of the algorithm theoretical basis for each of the GLAS data products. This will be the final version of this document. The algorithms were initially designed and written based on the authors prior experience with high altitude lidar data on systems such as the Cloud and Aerosol Lidar System (CALS) and the Cloud Physics Lidar (CPL), both of which fly on the NASA ER-2 high altitude aircraft. These lidar systems have been employed in many field experiments around the world and algorithms have been developed to analyze these data for a number of atmospheric parameters. CALS data have been analyzed for cloud top height, thin cloud optical depth, cirrus cloud emittance (Spinhirne and Hart, 1990) and boundary layer depth (Palm and Spinhirne, 1987, 1998). The successor to CALS, the CPL, has also been extensively deployed in field missions since 2000 including the validation of GLAS and CALIPSO. The CALS and early CPL data sets also served as the basis for the construction of simulated GLAS data sets which were then used to develop and test the GLAS analysis algorithms.

Global Distribution and Vertical Structure of Clouds Revealed by CALIPSO

NASA Astrophysics Data System (ADS)

Yi, Y.; Minnis, P.; Winker, D.; Huang, J.; Sun-Mack, S.; Ayers, K.

2007-12-01

Understanding the effects of clouds on Earth's radiation balance, especially on longwave fluxes within the atmosphere, depends on having accurate knowledge of cloud vertical location within the atmosphere. The Cloud- Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) satellite mission provides the opportunity to measure the vertical distribution of clouds at a greater detail than ever before possible. The CALIPSO cloud layer products from June 2006 to June 2007 are analyzed to determine the occurrence frequency and thickness of clouds as functions of time, latitude, and altitude. In particular, the latitude-longitude and vertical distributions of single- and multi-layer clouds and the latitudinal movement of cloud cover with the changing seasons are examined. The seasonal variablities of cloud frequency and geometric thickness are also analyzed and compared with similar quantities derived from the Aqua Moderate Resolution Imaging Spectroradiometer (MODIS) using the Clouds and the Earth's Radiant Energy System (CERES) cloud retrieval algorithms. The comparisons provide an estimate of the errors in cloud fraction, top height, and thickness incurred by passive algorithms.

Motion data classification on the basis of dynamic time warping with a cloud point distance measure

NASA Astrophysics Data System (ADS)

Switonski, Adam; Josinski, Henryk; Zghidi, Hafedh; Wojciechowski, Konrad

2016-06-01

The paper deals with the problem of classification of model free motion data. The nearest neighbors classifier which is based on comparison performed by Dynamic Time Warping transform with cloud point distance measure is proposed. The classification utilizes both specific gait features reflected by a movements of subsequent skeleton joints and anthropometric data. To validate proposed approach human gait identification challenge problem is taken into consideration. The motion capture database containing data of 30 different humans collected in Human Motion Laboratory of Polish-Japanese Academy of Information Technology is used. The achieved results are satisfactory, the obtained accuracy of human recognition exceeds 90%. What is more, the applied cloud point distance measure does not depend on calibration process of motion capture system which results in reliable validation.

A particle swarm optimized kernel-based clustering method for crop mapping from multi-temporal polarimetric L-band SAR observations

NASA Astrophysics Data System (ADS)

Tamiminia, Haifa; Homayouni, Saeid; McNairn, Heather; Safari, Abdoreza

2017-06-01

Polarimetric Synthetic Aperture Radar (PolSAR) data, thanks to their specific characteristics such as high resolution, weather and daylight independence, have become a valuable source of information for environment monitoring and management. The discrimination capability of observations acquired by these sensors can be used for land cover classification and mapping. The aim of this paper is to propose an optimized kernel-based C-means clustering algorithm for agriculture crop mapping from multi-temporal PolSAR data. Firstly, several polarimetric features are extracted from preprocessed data. These features are linear polarization intensities, and several statistical and physical based decompositions such as Cloude-Pottier, Freeman-Durden and Yamaguchi techniques. Then, the kernelized version of hard and fuzzy C-means clustering algorithms are applied to these polarimetric features in order to identify crop types. The kernel function, unlike the conventional partitioning clustering algorithms, simplifies the non-spherical and non-linearly patterns of data structure, to be clustered easily. In addition, in order to enhance the results, Particle Swarm Optimization (PSO) algorithm is used to tune the kernel parameters, cluster centers and to optimize features selection. The efficiency of this method was evaluated by using multi-temporal UAVSAR L-band images acquired over an agricultural area near Winnipeg, Manitoba, Canada, during June and July in 2012. The results demonstrate more accurate crop maps using the proposed method when compared to the classical approaches, (e.g. 12% improvement in general). In addition, when the optimization technique is used, greater improvement is observed in crop classification, e.g. 5% in overall. Furthermore, a strong relationship between Freeman-Durden volume scattering component, which is related to canopy structure, and phenological growth stages is observed.

Approach for Text Classification Based on the Similarity Measurement between Normal Cloud Models

PubMed Central

Dai, Jin; Liu, Xin

2014-01-01

The similarity between objects is the core research area of data mining. In order to reduce the interference of the uncertainty of nature language, a similarity measurement between normal cloud models is adopted to text classification research. On this basis, a novel text classifier based on cloud concept jumping up (CCJU-TC) is proposed. It can efficiently accomplish conversion between qualitative concept and quantitative data. Through the conversion from text set to text information table based on VSM model, the text qualitative concept, which is extraction from the same category, is jumping up as a whole category concept. According to the cloud similarity between the test text and each category concept, the test text is assigned to the most similar category. By the comparison among different text classifiers in different feature selection set, it fully proves that not only does CCJU-TC have a strong ability to adapt to the different text features, but also the classification performance is also better than the traditional classifiers. PMID:24711737

Fast Semantic Segmentation of 3d Point Clouds with Strongly Varying Density

NASA Astrophysics Data System (ADS)

Hackel, Timo; Wegner, Jan D.; Schindler, Konrad

2016-06-01

We describe an effective and efficient method for point-wise semantic classification of 3D point clouds. The method can handle unstructured and inhomogeneous point clouds such as those derived from static terrestrial LiDAR or photogammetric reconstruction; and it is computationally efficient, making it possible to process point clouds with many millions of points in a matter of minutes. The key issue, both to cope with strong variations in point density and to bring down computation time, turns out to be careful handling of neighborhood relations. By choosing appropriate definitions of a point's (multi-scale) neighborhood, we obtain a feature set that is both expressive and fast to compute. We evaluate our classification method both on benchmark data from a mobile mapping platform and on a variety of large, terrestrial laser scans with greatly varying point density. The proposed feature set outperforms the state of the art with respect to per-point classification accuracy, while at the same time being much faster to compute.

Enabling Earth Science Through Cloud Computing

NASA Technical Reports Server (NTRS)

Hardman, Sean; Riofrio, Andres; Shams, Khawaja; Freeborn, Dana; Springer, Paul; Chafin, Brian

2012-01-01

Cloud Computing holds tremendous potential for missions across the National Aeronautics and Space Administration. Several flight missions are already benefiting from an investment in cloud computing for mission critical pipelines and services through faster processing time, higher availability, and drastically lower costs available on cloud systems. However, these processes do not currently extend to general scientific algorithms relevant to earth science missions. The members of the Airborne Cloud Computing Environment task at the Jet Propulsion Laboratory have worked closely with the Carbon in Arctic Reservoirs Vulnerability Experiment (CARVE) mission to integrate cloud computing into their science data processing pipeline. This paper details the efforts involved in deploying a science data system for the CARVE mission, evaluating and integrating cloud computing solutions with the system and porting their science algorithms for execution in a cloud environment.

Standoff detection of bioaerosols over wide area using a newly developed sensor combining a cloud mapper and a spectrometric LIF lidar

NASA Astrophysics Data System (ADS)

Buteau, Sylvie; Simard, Jean-Robert; Roy, Gilles; Lahaie, Pierre; Nadeau, Denis; Mathieu, Pierre

2013-10-01

A standoff sensor called BioSense was developed to demonstrate the capacity to map, track and classify bioaerosol clouds from a distant range and over wide area. The concept of the system is based on a two steps dynamic surveillance: 1) cloud detection using an infrared (IR) scanning cloud mapper and 2) cloud classification based on a staring ultraviolet (UV) Laser Induced Fluorescence (LIF) interrogation. The system can be operated either in an automatic surveillance mode or using manual intervention. The automatic surveillance operation includes several steps: mission planning, sensor deployment, background monitoring, surveillance, cloud detection, classification and finally alarm generation based on the classification result. One of the main challenges is the classification step which relies on a spectrally resolved UV LIF signature library. The construction of this library relies currently on in-chamber releases of various materials that are simultaneously characterized with the standoff sensor and referenced with point sensors such as Aerodynamic Particle Sizer® (APS). The system was tested at three different locations in order to evaluate its capacity to operate in diverse types of surroundings and various environmental conditions. The system showed generally good performances even though the troubleshooting of the system was not completed before initiating the Test and Evaluation (T&E) process. The standoff system performances appeared to be highly dependent on the type of challenges, on the climatic conditions and on the period of day. The real-time results combined with the experience acquired during the 2012 T & E allowed to identify future ameliorations and investigation avenues.

Cloud-Scale Genomic Signals Processing for Robust Large-Scale Cancer Genomic Microarray Data Analysis.

PubMed

Harvey, Benjamin Simeon; Ji, Soo-Yeon

2017-01-01

As microarray data available to scientists continues to increase in size and complexity, it has become overwhelmingly important to find multiple ways to bring forth oncological inference to the bioinformatics community through the analysis of large-scale cancer genomic (LSCG) DNA and mRNA microarray data that is useful to scientists. Though there have been many attempts to elucidate the issue of bringing forth biological interpretation by means of wavelet preprocessing and classification, there has not been a research effort that focuses on a cloud-scale distributed parallel (CSDP) separable 1-D wavelet decomposition technique for denoising through differential expression thresholding and classification of LSCG microarray data. This research presents a novel methodology that utilizes a CSDP separable 1-D method for wavelet-based transformation in order to initialize a threshold which will retain significantly expressed genes through the denoising process for robust classification of cancer patients. Additionally, the overall study was implemented and encompassed within CSDP environment. The utilization of cloud computing and wavelet-based thresholding for denoising was used for the classification of samples within the Global Cancer Map, Cancer Cell Line Encyclopedia, and The Cancer Genome Atlas. The results proved that separable 1-D parallel distributed wavelet denoising in the cloud and differential expression thresholding increased the computational performance and enabled the generation of higher quality LSCG microarray datasets, which led to more accurate classification results.

Neural network cloud top pressure and height for MODIS

NASA Astrophysics Data System (ADS)

Håkansson, Nina; Adok, Claudia; Thoss, Anke; Scheirer, Ronald; Hörnquist, Sara

2018-06-01

Cloud top height retrieval from imager instruments is important for nowcasting and for satellite climate data records. A neural network approach for cloud top height retrieval from the imager instrument MODIS (Moderate Resolution Imaging Spectroradiometer) is presented. The neural networks are trained using cloud top layer pressure data from the CALIOP (Cloud-Aerosol Lidar with Orthogonal Polarization) dataset. Results are compared with two operational reference algorithms for cloud top height: the MODIS Collection 6 Level 2 height product and the cloud top temperature and height algorithm in the 2014 version of the NWC SAF (EUMETSAT (European Organization for the Exploitation of Meteorological Satellites) Satellite Application Facility on Support to Nowcasting and Very Short Range Forecasting) PPS (Polar Platform System). All three techniques are evaluated using both CALIOP and CPR (Cloud Profiling Radar for CloudSat (CLOUD SATellite)) height. Instruments like AVHRR (Advanced Very High Resolution Radiometer) and VIIRS (Visible Infrared Imaging Radiometer Suite) contain fewer channels useful for cloud top height retrievals than MODIS, therefore several different neural networks are investigated to test how infrared channel selection influences retrieval performance. Also a network with only channels available for the AVHRR1 instrument is trained and evaluated. To examine the contribution of different variables, networks with fewer variables are trained. It is shown that variables containing imager information for neighboring pixels are very important. The error distributions of the involved cloud top height algorithms are found to be non-Gaussian. Different descriptive statistic measures are presented and it is exemplified that bias and SD (standard deviation) can be misleading for non-Gaussian distributions. The median and mode are found to better describe the tendency of the error distributions and IQR (interquartile range) and MAE (mean absolute error) are found to give the most useful information of the spread of the errors. For all descriptive statistics presented MAE, IQR, RMSE (root mean square error), SD, mode, median, bias and percentage of absolute errors above 0.25, 0.5, 1 and 2 km the neural network perform better than the reference algorithms both validated with CALIOP and CPR (CloudSat). The neural networks using the brightness temperatures at 11 and 12 µm show at least 32 % (or 623 m) lower MAE compared to the two operational reference algorithms when validating with CALIOP height. Validation with CPR (CloudSat) height gives at least 25 % (or 430 m) reduction of MAE.

Spatial and Temporal Varying Thresholds for Cloud Detection in Satellite Imagery

NASA Technical Reports Server (NTRS)

Jedlovec, Gary; Haines, Stephanie

2007-01-01

A new cloud detection technique has been developed and applied to both geostationary and polar orbiting satellite imagery having channels in the thermal infrared and short wave infrared spectral regions. The bispectral composite threshold (BCT) technique uses only the 11 micron and 3.9 micron channels, and composite imagery generated from these channels, in a four-step cloud detection procedure to produce a binary cloud mask at single pixel resolution. A unique aspect of this algorithm is the use of 20-day composites of the 11 micron and the 11 - 3.9 micron channel difference imagery to represent spatially and temporally varying clear-sky thresholds for the bispectral cloud tests. The BCT cloud detection algorithm has been applied to GOES and MODIS data over the continental United States over the last three years with good success. The resulting products have been validated against "truth" datasets (generated by the manual determination of the sky conditions from available satellite imagery) for various seasons from the 2003-2005 periods. The day and night algorithm has been shown to determine the correct sky conditions 80-90% of the time (on average) over land and ocean areas. Only a small variation in algorithm performance occurs between day-night, land-ocean, and between seasons. The algorithm performs least well. during he winter season with only 80% of the sky conditions determined correctly. The algorithm was found to under-determine clouds at night and during times of low sun angle (in geostationary satellite data) and tends to over-determine the presence of clouds during the day, particularly in the summertime. Since the spectral tests use only the short- and long-wave channels common to most multispectral scanners; the application of the BCT technique to a variety of satellite sensors including SEVERI should be straightforward and produce similar performance results.

Hierarchical trie packet classification algorithm based on expectation-maximization clustering.

PubMed

Bi, Xia-An; Zhao, Junxia

2017-01-01

With the development of computer network bandwidth, packet classification algorithms which are able to deal with large-scale rule sets are in urgent need. Among the existing algorithms, researches on packet classification algorithms based on hierarchical trie have become an important packet classification research branch because of their widely practical use. Although hierarchical trie is beneficial to save large storage space, it has several shortcomings such as the existence of backtracking and empty nodes. This paper proposes a new packet classification algorithm, Hierarchical Trie Algorithm Based on Expectation-Maximization Clustering (HTEMC). Firstly, this paper uses the formalization method to deal with the packet classification problem by means of mapping the rules and data packets into a two-dimensional space. Secondly, this paper uses expectation-maximization algorithm to cluster the rules based on their aggregate characteristics, and thereby diversified clusters are formed. Thirdly, this paper proposes a hierarchical trie based on the results of expectation-maximization clustering. Finally, this paper respectively conducts simulation experiments and real-environment experiments to compare the performances of our algorithm with other typical algorithms, and analyzes the results of the experiments. The hierarchical trie structure in our algorithm not only adopts trie path compression to eliminate backtracking, but also solves the problem of low efficiency of trie updates, which greatly improves the performance of the algorithm.

Marine Boundary Layer Cloud Property Retrievals from High-Resolution ASTER Observations: Case Studies and Comparison with Terra MODIS

NASA Technical Reports Server (NTRS)

Werner, Frank; Wind, Galina; Zhang, Zhibo; Platnick, Steven; Di Girolamo, Larry; Zhao, Guangyu; Amarasinghe, Nandana; Meyer, Kerry

2016-01-01

A research-level retrieval algorithm for cloud optical and microphysical properties is developed for the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) aboard the Terra satellite. It is based on the operational MODIS algorithm. This paper documents the technical details of this algorithm and evaluates the retrievals for selected marine boundary layer cloud scenes through comparisons with the operational MODIS Data Collection 6 (C6) cloud product. The newly developed, ASTERspecific cloud masking algorithm is evaluated through comparison with an independent algorithm reported in Zhao and Di Girolamo (2006). To validate and evaluate the cloud optical thickness (tau) and cloud effective radius (r(sub eff)) from ASTER, the high-spatial-resolution ASTER observations are first aggregated to the same 1000m resolution as MODIS. Subsequently, tau(sub aA) and r(sub eff, aA) retrieved from the aggregated ASTER radiances are compared with the collocated MODIS retrievals. For overcast pixels, the two data sets agree very well with Pearson's product-moment correlation coefficients of R greater than 0.970. However, for partially cloudy pixels there are significant differences between r(sub eff, aA) and the MODIS results which can exceed 10 micrometers. Moreover, it is shown that the numerous delicate cloud structures in the example marine boundary layer scenes, resolved by the high-resolution ASTER retrievals, are smoothed by the MODIS observations. The overall good agreement between the research-level ASTER results and the operational MODIS C6 products proves the feasibility of MODIS-like retrievals from ASTER reflectance measurements and provides the basis for future studies concerning the scale dependency of satellite observations and three-dimensional radiative effects.

Marine boundary layer cloud property retrievals from high-resolution ASTER observations: case studies and comparison with Terra MODIS

NASA Astrophysics Data System (ADS)

Werner, Frank; Wind, Galina; Zhang, Zhibo; Platnick, Steven; Di Girolamo, Larry; Zhao, Guangyu; Amarasinghe, Nandana; Meyer, Kerry

2016-12-01

A research-level retrieval algorithm for cloud optical and microphysical properties is developed for the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) aboard the Terra satellite. It is based on the operational MODIS algorithm. This paper documents the technical details of this algorithm and evaluates the retrievals for selected marine boundary layer cloud scenes through comparisons with the operational MODIS Data Collection 6 (C6) cloud product. The newly developed, ASTER-specific cloud masking algorithm is evaluated through comparison with an independent algorithm reported in [Zhao and Di Girolamo(2006)]. To validate and evaluate the cloud optical thickness (τ) and cloud effective radius (reff) from ASTER, the high-spatial-resolution ASTER observations are first aggregated to the same 1000 m resolution as MODIS. Subsequently, τaA and reff, aA retrieved from the aggregated ASTER radiances are compared with the collocated MODIS retrievals. For overcast pixels, the two data sets agree very well with Pearson's product-moment correlation coefficients of R > 0.970. However, for partially cloudy pixels there are significant differences between reff, aA and the MODIS results which can exceed 10 µm. Moreover, it is shown that the numerous delicate cloud structures in the example marine boundary layer scenes, resolved by the high-resolution ASTER retrievals, are smoothed by the MODIS observations. The overall good agreement between the research-level ASTER results and the operational MODIS C6 products proves the feasibility of MODIS-like retrievals from ASTER reflectance measurements and provides the basis for future studies concerning the scale dependency of satellite observations and three-dimensional radiative effects.

Global aerosol typing from a combination of A-Train satellite observations in clear-sky and above clouds

NASA Astrophysics Data System (ADS)

Kacenelenbogen, M. S.; Russell, P. B.; Vaughan, M.; Redemann, J.; Shinozuka, Y.; Livingston, J. M.; Zhang, Q.

2014-12-01

According to the 5th Assessment Report of the Intergovernmental Panel on Climate Change (IPCC), the model estimates of Radiative Forcing due to aerosol-radiation interactions (RFari) for individual aerosol types are less certain than the total RFari [Boucher et al., 2013]. For example, the RFari specific to Black Carbon (BC) is uncertain due to an underestimation of its mass concentration near source regions [Koch et al., 2009]. Several recent studies have evaluated chemical transport model (CTM) predictions using observations of aerosol optical properties such as Aerosol Optical Depth (AOD) or Single Scattering Albedo (SSA) from satellite or ground-based instruments (e.g., Huneeus et al., [2010]). However, most passive remote sensing instruments fail to provide a comprehensive assessment of the particle type without further analysis and combination of measurements. To improve the predictions of aerosol composition in CTMs, we have developed an aerosol classification algorithm (called Specified Clustering and Mahalanobis Classification, SCMC) that assigns an aerosol type to multi-parameter retrievals by spaceborne, airborne or ground based passive remote sensing instruments [Russell et al., 2014]. The aerosol types identified by our scheme are pure dust, polluted dust, urban-industrial/developed economy, urban-industrial/developing economy, dark biomass smoke, light biomass smoke and pure marine. First, we apply the SCMC method to five years of clear-sky space-borne POLDER observations over Greece. We then use the aerosol extinction and SSA spectra retrieved from a combination of MODIS, OMI and CALIOP clear-sky observations to infer the aerosol type over the globe in 2007. Finally, we will extend the spaceborne aerosol classification from clear-sky to above low opaque water clouds using a combination of CALIOP AOD and backscatter observations and OMI absorption AOD values from near-by clear-sky pixels.

V2.2 L2AS Detailed Release Description April 15, 2002

Atmospheric Science Data Center

2013-03-14

... 'optically thick atmosphere' algorithm. Implement new experimental aerosol retrieval algorithm over homogeneous surface types. ... Change values: cloud_mask_decision_matrix(1,1): .true. -> .false. cloud_mask_decision_matrix(2,1): .true. -> .false. ...

A simple biota removal algorithm for 35 GHz cloud radar measurements

NASA Astrophysics Data System (ADS)

Kalapureddy, Madhu Chandra R.; Sukanya, Patra; Das, Subrata K.; Deshpande, Sachin M.; Pandithurai, Govindan; Pazamany, Andrew L.; Ambuj K., Jha; Chakravarty, Kaustav; Kalekar, Prasad; Krishna Devisetty, Hari; Annam, Sreenivas

2018-03-01

Cloud radar reflectivity profiles can be an important measurement for the investigation of cloud vertical structure (CVS). However, extracting intended meteorological cloud content from the measurement often demands an effective technique or algorithm that can reduce error and observational uncertainties in the recorded data. In this work, a technique is proposed to identify and separate cloud and non-hydrometeor echoes using the radar Doppler spectral moments profile measurements. The point and volume target-based theoretical radar sensitivity curves are used for removing the receiver noise floor and identified radar echoes are scrutinized according to the signal decorrelation period. Here, it is hypothesized that cloud echoes are observed to be temporally more coherent and homogenous and have a longer correlation period than biota. That can be checked statistically using ˜ 4 s sliding mean and standard deviation value of reflectivity profiles. The above step helps in screen out clouds critically by filtering out the biota. The final important step strives for the retrieval of cloud height. The proposed algorithm potentially identifies cloud height solely through the systematic characterization of Z variability using the local atmospheric vertical structure knowledge besides to the theoretical, statistical and echo tracing tools. Thus, characterization of high-resolution cloud radar reflectivity profile measurements has been done with the theoretical echo sensitivity curves and observed echo statistics for the true cloud height tracking (TEST). TEST showed superior performance in screening out clouds and filtering out isolated insects. TEST constrained with polarimetric measurements was found to be more promising under high-density biota whereas TEST combined with linear depolarization ratio and spectral width perform potentially to filter out biota within the highly turbulent shallow cumulus clouds in the convective boundary layer (CBL). This TEST technique is promisingly simple in realization but powerful in performance due to the flexibility in constraining, identifying and filtering out the biota and screening out the true cloud content, especially the CBL clouds. Therefore, the TEST algorithm is superior for screening out the low-level clouds that are strongly linked to the rainmaking mechanism associated with the Indian Summer Monsoon region's CVS.

Cloud Detection with the Earth Polychromatic Imaging Camera (EPIC)

NASA Technical Reports Server (NTRS)

Meyer, Kerry; Marshak, Alexander; Lyapustin, Alexei; Torres, Omar; Wang, Yugie

2011-01-01

The Earth Polychromatic Imaging Camera (EPIC) on board the Deep Space Climate Observatory (DSCOVR) would provide a unique opportunity for Earth and atmospheric research due not only to its Lagrange point sun-synchronous orbit, but also to the potential for synergistic use of spectral channels in both the UV and visible spectrum. As a prerequisite for most applications, the ability to detect the presence of clouds in a given field of view, known as cloud masking, is of utmost importance. It serves to determine both the potential for cloud contamination in clear-sky applications (e.g., land surface products and aerosol retrievals) and clear-sky contamination in cloud applications (e.g., cloud height and property retrievals). To this end, a preliminary cloud mask algorithm has been developed for EPIC that applies thresholds to reflected UV and visible radiances, as well as to reflected radiance ratios. This algorithm has been tested with simulated EPIC radiances over both land and ocean scenes, with satisfactory results. These test results, as well as algorithm sensitivity to potential instrument uncertainties, will be presented.

Satellite Imagery Analysis for Nighttime Temperature Inversion Clouds

NASA Technical Reports Server (NTRS)

Kawamoto, K.; Minnis, P.; Arduini, R.; Smith, W., Jr.

2001-01-01

Clouds play important roles in the climate system. Their optical and microphysical properties, which largely determine their radiative property, need to be investigated. Among several measurement means, satellite remote sensing seems to be the most promising. Since most of the cloud algorithms proposed so far are daytime use which utilizes solar radiation, Minnis et al. (1998) developed a nighttime use one using 3.7-, 11 - and 12-microns channels. Their algorithm, however, has a drawback that is not able to treat temperature inversion cases. We update their algorithm, incorporating new parameterization by Arduini et al. (1999) which is valid for temperature inversion cases. This updated algorithm has been applied to GOES satellite data and reasonable retrieval results were obtained.

Comparison of the MODIS Collection 5 Multilayer Cloud Detection Product with CALIPSO

NASA Technical Reports Server (NTRS)

Platnick, Steven; Wind, Gala; King, Michael D.; Holz, Robert E.; Ackerman, Steven A.; Nagle, Fred W.

2010-01-01

CALIPSO, launched in June 2006, provides global active remote sensing measurements of clouds and aerosols that can be used for validation of a variety of passive imager retrievals derived from instruments flying on the Aqua spacecraft and other A-Train platforms. The most recent processing effort for the MODIS Atmosphere Team, referred to as the Collection 5 scream, includes a research-level multilayer cloud detection algorithm that uses both thermodynamic phase information derived from a combination of solar and thermal emission bands to discriminate layers of different phases, as well as true layer separation discrimination using a moderately absorbing water vapor band. The multilayer detection algorithm is designed to provide a means of assessing the applicability of 1D cloud models used in the MODIS cloud optical and microphysical product retrieval, which are generated at a 1 km resolution. Using pixel-level collocations of MODIS Aqua, CALIOP, we investigate the global performance of multilayer cloud detection algorithms (and thermodynamic phase).

The GRAPE aerosol retrieval algorithm

NASA Astrophysics Data System (ADS)

Thomas, G. E.; Poulsen, C. A.; Sayer, A. M.; Marsh, S. H.; Dean, S. M.; Carboni, E.; Siddans, R.; Grainger, R. G.; Lawrence, B. N.

2009-11-01

The aerosol component of the Oxford-Rutherford Aerosol and Cloud (ORAC) combined cloud and aerosol retrieval scheme is described and the theoretical performance of the algorithm is analysed. ORAC is an optimal estimation retrieval scheme for deriving cloud and aerosol properties from measurements made by imaging satellite radiometers and, when applied to cloud free radiances, provides estimates of aerosol optical depth at a wavelength of 550 nm, aerosol effective radius and surface reflectance at 550 nm. The aerosol retrieval component of ORAC has several incarnations - this paper addresses the version which operates in conjunction with the cloud retrieval component of ORAC (described by Watts et al., 1998), as applied in producing the Global Retrieval of ATSR Cloud Parameters and Evaluation (GRAPE) data-set. The algorithm is described in detail and its performance examined. This includes a discussion of errors resulting from the formulation of the forward model, sensitivity of the retrieval to the measurements and a priori constraints, and errors resulting from assumptions made about the atmospheric/surface state.

The GRAPE aerosol retrieval algorithm

NASA Astrophysics Data System (ADS)

Thomas, G. E.; Poulsen, C. A.; Sayer, A. M.; Marsh, S. H.; Dean, S. M.; Carboni, E.; Siddans, R.; Grainger, R. G.; Lawrence, B. N.

2009-04-01

The aerosol component of the Oxford-Rutherford Aerosol and Cloud (ORAC) combined cloud and aerosol retrieval scheme is described and the theoretical performance of the algorithm is analysed. ORAC is an optimal estimation retrieval scheme for deriving cloud and aerosol properties from measurements made by imaging satellite radiometers and, when applied to cloud free radiances, provides estimates of aerosol optical depth at a wavelength of 550 nm, aerosol effective radius and surface reflectance at 550 nm. The aerosol retrieval component of ORAC has several incarnations - this paper addresses the version which operates in conjunction with the cloud retrieval component of ORAC (described by Watts et al., 1998), as applied in producing the Global Retrieval of ATSR Cloud Parameters and Evaluation (GRAPE) data-set. The algorithm is described in detail and its performance examined. This includes a discussion of errors resulting from the formulation of the forward model, sensitivity of the retrieval to the measurements and a priori constraints, and errors resulting from assumptions made about the atmospheric/surface state.

Tree Classification with Fused Mobile Laser Scanning and Hyperspectral Data

PubMed Central

Puttonen, Eetu; Jaakkola, Anttoni; Litkey, Paula; Hyyppä, Juha

2011-01-01

Mobile Laser Scanning data were collected simultaneously with hyperspectral data using the Finnish Geodetic Institute Sensei system. The data were tested for tree species classification. The test area was an urban garden in the City of Espoo, Finland. Point clouds representing 168 individual tree specimens of 23 tree species were determined manually. The classification of the trees was done using first only the spatial data from point clouds, then with only the spectral data obtained with a spectrometer, and finally with the combined spatial and hyperspectral data from both sensors. Two classification tests were performed: the separation of coniferous and deciduous trees, and the identification of individual tree species. All determined tree specimens were used in distinguishing coniferous and deciduous trees. A subset of 133 trees and 10 tree species was used in the tree species classification. The best classification results for the fused data were 95.8% for the separation of the coniferous and deciduous classes. The best overall tree species classification succeeded with 83.5% accuracy for the best tested fused data feature combination. The respective results for paired structural features derived from the laser point cloud were 90.5% for the separation of the coniferous and deciduous classes and 65.4% for the species classification. Classification accuracies with paired hyperspectral reflectance value data were 90.5% for the separation of coniferous and deciduous classes and 62.4% for different species. The results are among the first of their kind and they show that mobile collected fused data outperformed single-sensor data in both classification tests and by a significant margin. PMID:22163894

Tree classification with fused mobile laser scanning and hyperspectral data.

PubMed

Puttonen, Eetu; Jaakkola, Anttoni; Litkey, Paula; Hyyppä, Juha

2011-01-01

Mobile Laser Scanning data were collected simultaneously with hyperspectral data using the Finnish Geodetic Institute Sensei system. The data were tested for tree species classification. The test area was an urban garden in the City of Espoo, Finland. Point clouds representing 168 individual tree specimens of 23 tree species were determined manually. The classification of the trees was done using first only the spatial data from point clouds, then with only the spectral data obtained with a spectrometer, and finally with the combined spatial and hyperspectral data from both sensors. Two classification tests were performed: the separation of coniferous and deciduous trees, and the identification of individual tree species. All determined tree specimens were used in distinguishing coniferous and deciduous trees. A subset of 133 trees and 10 tree species was used in the tree species classification. The best classification results for the fused data were 95.8% for the separation of the coniferous and deciduous classes. The best overall tree species classification succeeded with 83.5% accuracy for the best tested fused data feature combination. The respective results for paired structural features derived from the laser point cloud were 90.5% for the separation of the coniferous and deciduous classes and 65.4% for the species classification. Classification accuracies with paired hyperspectral reflectance value data were 90.5% for the separation of coniferous and deciduous classes and 62.4% for different species. The results are among the first of their kind and they show that mobile collected fused data outperformed single-sensor data in both classification tests and by a significant margin.

Atrial Fibrillation Screening in Nonmetropolitan Areas Using a Telehealth Surveillance System With an Embedded Cloud-Computing Algorithm: Prospective Pilot Study

PubMed Central

Chen, Ying-Hsien; Hung, Chi-Sheng; Huang, Ching-Chang; Hung, Yu-Chien

2017-01-01

Background Atrial fibrillation (AF) is a common form of arrhythmia that is associated with increased risk of stroke and mortality. Detecting AF before the first complication occurs is a recognized priority. No previous studies have examined the feasibility of undertaking AF screening using a telehealth surveillance system with an embedded cloud-computing algorithm; we address this issue in this study. Objective The objective of this study was to evaluate the feasibility of AF screening in nonmetropolitan areas using a telehealth surveillance system with an embedded cloud-computing algorithm. Methods We conducted a prospective AF screening study in a nonmetropolitan area using a single-lead electrocardiogram (ECG) recorder. All ECG measurements were reviewed on the telehealth surveillance system and interpreted by the cloud-computing algorithm and a cardiologist. The process of AF screening was evaluated with a satisfaction questionnaire. Results Between March 11, 2016 and August 31, 2016, 967 ECGs were recorded from 922 residents in nonmetropolitan areas. A total of 22 (2.4%, 22/922) residents with AF were identified by the physician’s ECG interpretation, and only 0.2% (2/967) of ECGs contained significant artifacts. The novel cloud-computing algorithm for AF detection had a sensitivity of 95.5% (95% CI 77.2%-99.9%) and specificity of 97.7% (95% CI 96.5%-98.5%). The overall satisfaction score for the process of AF screening was 92.1%. Conclusions AF screening in nonmetropolitan areas using a telehealth surveillance system with an embedded cloud-computing algorithm is feasible. PMID:28951384

A smartphone application for psoriasis segmentation and classification (Conference Presentation)

NASA Astrophysics Data System (ADS)

Vasefi, Fartash; MacKinnon, Nicholas B.; Horita, Timothy; Shi, Kevin; Khan Munia, Tamanna Tabassum; Tavakolian, Kouhyar; Alhashim, Minhal; Fazel-Rezai, Reza

2017-02-01

Psoriasis is a chronic skin disease affecting approximately 125 million people worldwide. Currently, dermatologists monitor changes of psoriasis by clinical evaluation or by measuring psoriasis severity scores over time which lead to Subjective management of this condition. The goal of this paper is to develop a reliable assessment system to quantitatively assess the changes of erythema and intensity of scaling of psoriatic lesions. A smartphone deployable mobile application is presented that uses the smartphone camera and cloud-based image processing to analyze physiological characteristics of psoriasis lesions, identify the type and stage of the scaling and erythema. The application targets to automatically evaluate Psoriasis Area Severity Index (PASI) by measuring the severity and extent of psoriasis. The mobile application performs the following core functions: 1) it captures text information from user input to create a profile in a HIPAA compliant database. 2) It captures an image of the skin with psoriasis as well as image-related information entered by the user. 3) The application color correct the image based on environmental lighting condition using calibration process including calibration procedure by capturing Macbeth ColorChecker image. 4) The color-corrected image will be transmitted to a cloud-based engine for image processing. In cloud, first, the algorithm removes the non-skin background to ensure the psoriasis segmentation is only applied to the skin regions. Then, the psoriasis segmentation algorithm estimates the erythema and scaling boundary regions of lesion. We analyzed 10 images of psoriasis images captured by cellphone, determined PASI score for each subject during our pilot study, and correlated it with changes in severity scores given by dermatologists. The success of this work allows smartphone application for psoriasis severity assessment in a long-term treatment.

Assessing the Performance of a Machine Learning Algorithm in Identifying Bubbles in Dust Emission

NASA Astrophysics Data System (ADS)

Xu, Duo; Offner, Stella S. R.

2017-12-01

Stellar feedback created by radiation and winds from massive stars plays a significant role in both physical and chemical evolution of molecular clouds. This energy and momentum leaves an identifiable signature (“bubbles”) that affects the dynamics and structure of the cloud. Most bubble searches are performed “by eye,” which is usually time-consuming, subjective, and difficult to calibrate. Automatic classifications based on machine learning make it possible to perform systematic, quantifiable, and repeatable searches for bubbles. We employ a previously developed machine learning algorithm, Brut, and quantitatively evaluate its performance in identifying bubbles using synthetic dust observations. We adopt magnetohydrodynamics simulations, which model stellar winds launching within turbulent molecular clouds, as an input to generate synthetic images. We use a publicly available three-dimensional dust continuum Monte Carlo radiative transfer code, HYPERION, to generate synthetic images of bubbles in three Spitzer bands (4.5, 8, and 24 μm). We designate half of our synthetic bubbles as a training set, which we use to train Brut along with citizen-science data from the Milky Way Project (MWP). We then assess Brut’s accuracy using the remaining synthetic observations. We find that Brut’s performance after retraining increases significantly, and it is able to identify yellow bubbles, which are likely associated with B-type stars. Brut continues to perform well on previously identified high-score bubbles, and over 10% of the MWP bubbles are reclassified as high-confidence bubbles, which were previously marginal or ambiguous detections in the MWP data. We also investigate the influence of the size of the training set, dust model, evolutionary stage, and background noise on bubble identification.

Space Situational Awareness Data Processing Scalability Utilizing Google Cloud Services

NASA Astrophysics Data System (ADS)

Greenly, D.; Duncan, M.; Wysack, J.; Flores, F.

Space Situational Awareness (SSA) is a fundamental and critical component of current space operations. The term SSA encompasses the awareness, understanding and predictability of all objects in space. As the population of orbital space objects and debris increases, the number of collision avoidance maneuvers grows and prompts the need for accurate and timely process measures. The SSA mission continually evolves to near real-time assessment and analysis demanding the need for higher processing capabilities. By conventional methods, meeting these demands requires the integration of new hardware to keep pace with the growing complexity of maneuver planning algorithms. SpaceNav has implemented a highly scalable architecture that will track satellites and debris by utilizing powerful virtual machines on the Google Cloud Platform. SpaceNav algorithms for processing CDMs outpace conventional means. A robust processing environment for tracking data, collision avoidance maneuvers and various other aspects of SSA can be created and deleted on demand. Migrating SpaceNav tools and algorithms into the Google Cloud Platform will be discussed and the trials and tribulations involved. Information will be shared on how and why certain cloud products were used as well as integration techniques that were implemented. Key items to be presented are: 1.Scientific algorithms and SpaceNav tools integrated into a scalable architecture a) Maneuver Planning b) Parallel Processing c) Monte Carlo Simulations d) Optimization Algorithms e) SW Application Development/Integration into the Google Cloud Platform 2. Compute Engine Processing a) Application Engine Automated Processing b) Performance testing and Performance Scalability c) Cloud MySQL databases and Database Scalability d) Cloud Data Storage e) Redundancy and Availability

Evolutionary Multiobjective Query Workload Optimization of Cloud Data Warehouses

PubMed Central

Dokeroglu, Tansel; Sert, Seyyit Alper; Cinar, Muhammet Serkan

2014-01-01

With the advent of Cloud databases, query optimizers need to find paretooptimal solutions in terms of response time and monetary cost. Our novel approach minimizes both objectives by deploying alternative virtual resources and query plans making use of the virtual resource elasticity of the Cloud. We propose an exact multiobjective branch-and-bound and a robust multiobjective genetic algorithm for the optimization of distributed data warehouse query workloads on the Cloud. In order to investigate the effectiveness of our approach, we incorporate the devised algorithms into a prototype system. Finally, through several experiments that we have conducted with different workloads and virtual resource configurations, we conclude remarkable findings of alternative deployments as well as the advantages and disadvantages of the multiobjective algorithms we propose. PMID:24892048

a Gross Error Elimination Method for Point Cloud Data Based on Kd-Tree

NASA Astrophysics Data System (ADS)

Kang, Q.; Huang, G.; Yang, S.

2018-04-01

Point cloud data has been one type of widely used data sources in the field of remote sensing. Key steps of point cloud data's pro-processing focus on gross error elimination and quality control. Owing to the volume feature of point could data, existed gross error elimination methods need spend massive memory both in space and time. This paper employed a new method which based on Kd-tree algorithm to construct, k-nearest neighbor algorithm to search, settled appropriate threshold to determine with result turns out a judgement that whether target point is or not an outlier. Experimental results show that, our proposed algorithm will help to delete gross error in point cloud data and facilitate to decrease memory consumption, improve efficiency.

Testing a polarimetric cloud imager aboard research vessel Polarstern: comparison of color-based and polarimetric cloud detection algorithms.

PubMed

Barta, András; Horváth, Gábor; Horváth, Ákos; Egri, Ádám; Blahó, Miklós; Barta, Pál; Bumke, Karl; Macke, Andreas

2015-02-10

Cloud cover estimation is an important part of routine meteorological observations. Cloudiness measurements are used in climate model evaluation, nowcasting solar radiation, parameterizing the fluctuations of sea surface insolation, and building energy transfer models of the atmosphere. Currently, the most widespread ground-based method to measure cloudiness is based on analyzing the unpolarized intensity and color distribution of the sky obtained by digital cameras. As a new approach, we propose that cloud detection can be aided by the additional use of skylight polarization measured by 180° field-of-view imaging polarimetry. In the fall of 2010, we tested such a novel polarimetric cloud detector aboard the research vessel Polarstern during expedition ANT-XXVII/1. One of our goals was to test the durability of the measurement hardware under the extreme conditions of a trans-Atlantic cruise. Here, we describe the instrument and compare the results of several different cloud detection algorithms, some conventional and some newly developed. We also discuss the weaknesses of our design and its possible improvements. The comparison with cloud detection algorithms developed for traditional nonpolarimetric full-sky imagers allowed us to evaluate the added value of polarimetric quantities. We found that (1) neural-network-based algorithms perform the best among the investigated schemes and (2) global information (the mean and variance of intensity), nonoptical information (e.g., sun-view geometry), and polarimetric information (e.g., the degree of polarization) improve the accuracy of cloud detection, albeit slightly.

A High Performance Cloud-Based Protein-Ligand Docking Prediction Algorithm

PubMed Central

Chen, Jui-Le; Yang, Chu-Sing

2013-01-01

The potential of predicting druggability for a particular disease by integrating biological and computer science technologies has witnessed success in recent years. Although the computer science technologies can be used to reduce the costs of the pharmaceutical research, the computation time of the structure-based protein-ligand docking prediction is still unsatisfied until now. Hence, in this paper, a novel docking prediction algorithm, named fast cloud-based protein-ligand docking prediction algorithm (FCPLDPA), is presented to accelerate the docking prediction algorithm. The proposed algorithm works by leveraging two high-performance operators: (1) the novel migration (information exchange) operator is designed specially for cloud-based environments to reduce the computation time; (2) the efficient operator is aimed at filtering out the worst search directions. Our simulation results illustrate that the proposed method outperforms the other docking algorithms compared in this paper in terms of both the computation time and the quality of the end result. PMID:23762864

Feature-based three-dimensional registration for repetitive geometry in machine vision

PubMed Central

Gong, Yuanzheng; Seibel, Eric J.

2016-01-01

As an important step in three-dimensional (3D) machine vision, 3D registration is a process of aligning two or multiple 3D point clouds that are collected from different perspectives together into a complete one. The most popular approach to register point clouds is to minimize the difference between these point clouds iteratively by Iterative Closest Point (ICP) algorithm. However, ICP does not work well for repetitive geometries. To solve this problem, a feature-based 3D registration algorithm is proposed to align the point clouds that are generated by vision-based 3D reconstruction. By utilizing texture information of the object and the robustness of image features, 3D correspondences can be retrieved so that the 3D registration of two point clouds is to solve a rigid transformation. The comparison of our method and different ICP algorithms demonstrates that our proposed algorithm is more accurate, efficient and robust for repetitive geometry registration. Moreover, this method can also be used to solve high depth uncertainty problem caused by little camera baseline in vision-based 3D reconstruction. PMID:28286703

Spatial and Temporal Distribution of Clouds as Observed by MODIS Onboard the Terra and Aqua Satellites

NASA Technical Reports Server (NTRS)

King, Michael D.; Platnick, Steven; Menzel, Paul; Ackerman, Steven A.

2006-01-01

The Moderate Resolution Imaging Spectroradiometer (MODIS) was developed by NASA and launched onboard the Terra spacecraft on December 18,1999 and Aqua spacecraft on May 4, 2002. It achieved its final orbit and began Earth observations on February 24,2000 for Terra and June 24,2002 for Aqua. A comprehensive set of remote sensing algorithms for cloud masking and the retrieval of cloud physical and optical properties has been developed by members of the MODIS atmosphere science team. The archived products from these algorithms have applications in climate change studies, climate modeling, numerical weather prediction, and fundamental atmospheric research. In addition to an extensive cloud mask, products include cloud-top properties (temperature, pressure, effective emissivity), cloud thermodynamic phase, cloud optical and microphysical parameters (optical thickness, effective particle radius, water path), as well as derived statistics. Over the last year, extensive improvements and enhancements in the global cloud products have been implemented, and reprocessing of all MODIS data on Terra has commenced since first light in February 2000. In the cloud mask algorithm, the most extensive improvements were in distinguishing clouds at nighttime, including the challenging polar darkness regions of the world. Additional improvements have been made to properly distinguish sunglint from clouds in the tropical ocean regions, and to improve the identification of clouds from snow during daytime in Polar Regions. We will show global monthly mean cloud fraction for both Terra and Aqua, and show how similar the global daytime cloud fraction is from these morning and afternoon orbits, respectively. We will also show the zonal distribution of cloud fraction over land and ocean regions for both Terra and Aqua, and show the time series of global cloud fraction from July 2002 through June 2006.

Machine Learning Algorithms for Automatic Classification of Marmoset Vocalizations

PubMed Central

Ribeiro, Sidarta; Pereira, Danillo R.; Papa, João P.; de Albuquerque, Victor Hugo C.

2016-01-01

Automatic classification of vocalization type could potentially become a useful tool for acoustic the monitoring of captive colonies of highly vocal primates. However, for classification to be useful in practice, a reliable algorithm that can be successfully trained on small datasets is necessary. In this work, we consider seven different classification algorithms with the goal of finding a robust classifier that can be successfully trained on small datasets. We found good classification performance (accuracy > 0.83 and F1-score > 0.84) using the Optimum Path Forest classifier. Dataset and algorithms are made publicly available. PMID:27654941

Determination of effective droplet radius and optical depth of liquid water clouds over a tropical site in northern Thailand using passive microwave soundings, aircraft measurements and spectral irradiance data

NASA Astrophysics Data System (ADS)

Nimnuan, P.; Janjai, S.; Nunez, M.; Pratummasoot, N.; Buntoung, S.; Charuchittipan, D.; Chanyatham, T.; Chantraket, P.; Tantiplubthong, N.

2017-08-01

This paper presents an algorithm for deriving the effective droplet radius and optical depth of liquid water clouds using ground-based measurements, aircraft observations and an adiabatic model of cloud liquid water. The algorithm derives cloud effective radius and cloud optical depth over a tropical site at Omkoi (17.80°N, 98.43°E), Thailand. Monthly averages of cloud optical depth are highest in April (54.5), which is the month with the lowest average cloud effective radius (4.2 μm), both occurring before the start of the rainy season and at the end of the high contamination period. By contrast, the monsoon period extending from May to October brings higher cloud effective radius and lower cloud optical depth to the region on average. At the diurnal scale there is a gradual increase in average cloud optical depth and decrease in cloud effective radius as the day progresses.

Evaluating NASA S-NPP continuity cloud products for climate research using CALIPSO, CATS and Level-3 analysis

NASA Astrophysics Data System (ADS)

Holz, R.; Platnick, S. E.; Meyer, K.; Frey, R.; Wind, G.; Ackerman, S. A.; Heidinger, A. K.; Botambekov, D.; Yorks, J. E.; McGill, M. J.

2016-12-01

The launch of VIIRS and CrIS on Suomi NPP in the fall of 2011 introduced the next generation of U.S. operational polar orbiting environmental observations. Similar to MODIS, VIIRS provides visible and IR observations at moderate spatial resolution and has a 1:30 pm equatorial crossing time consistent with the MODIS on Aqua platform. However unlike MODIS, VIIRS lacks water vapor and CO2 absorbing channels that are used by the MODIS cloud algorithms for both cloud detection and to retrieve cloud top height and cloud emissivity for ice clouds. Given the different spectral and spatial characteristics of VIIRS, we seek to understand the extent to which the 15-year MODIS climate record can be continued with VIIRS/CrIS observations while maintaining consistent sensitivities across the observational systems. This presentation will focus on the evaluation of the latest version of the NASA funded cloud retrieval algorithms being developed for climate research. We will present collocated inter-comparisons between the imagers (VIIRS and MODIS Aqua) with CALIPSO and Cloud Aerosol Transport System (CATS) lidar observations as well as long term statistics based on a new Level-3 (L3) product being developed as part the project. The CALIPSO inter-comparisons will focus on cloud detection (cloud mask) with a focus on the impact of recent modifications to the cloud mask and how these changes impact the global statistics. For the first time we will provide inter-comparisons between two different cloud lidar systems (CALIOP and CATS) and investigate how the different sensitivities of the lidars impact the cloud mask and cloud comparisons. Using CALIPSO and CATS as the reference, and applying the same algorithms to VIIRS and MODIS, we will discuss the consistency between products from both imagers. The L3 analysis will focus on the regional and seasonal consistency between the suite of MODIS and VIIRS continuity cloud products. Do systematic biases remains when using consistent algorithms but applied to different observations (MODIS or VIIRS)?

Cloud computing for comparative genomics

PubMed Central

2010-01-01

Background Large comparative genomics studies and tools are becoming increasingly more compute-expensive as the number of available genome sequences continues to rise. The capacity and cost of local computing infrastructures are likely to become prohibitive with the increase, especially as the breadth of questions continues to rise. Alternative computing architectures, in particular cloud computing environments, may help alleviate this increasing pressure and enable fast, large-scale, and cost-effective comparative genomics strategies going forward. To test this, we redesigned a typical comparative genomics algorithm, the reciprocal smallest distance algorithm (RSD), to run within Amazon's Elastic Computing Cloud (EC2). We then employed the RSD-cloud for ortholog calculations across a wide selection of fully sequenced genomes. Results We ran more than 300,000 RSD-cloud processes within the EC2. These jobs were farmed simultaneously to 100 high capacity compute nodes using the Amazon Web Service Elastic Map Reduce and included a wide mix of large and small genomes. The total computation time took just under 70 hours and cost a total of $6,302 USD. Conclusions The effort to transform existing comparative genomics algorithms from local compute infrastructures is not trivial. However, the speed and flexibility of cloud computing environments provides a substantial boost with manageable cost. The procedure designed to transform the RSD algorithm into a cloud-ready application is readily adaptable to similar comparative genomics problems. PMID:20482786

Cloud computing for comparative genomics.

PubMed

Wall, Dennis P; Kudtarkar, Parul; Fusaro, Vincent A; Pivovarov, Rimma; Patil, Prasad; Tonellato, Peter J

2010-05-18

Large comparative genomics studies and tools are becoming increasingly more compute-expensive as the number of available genome sequences continues to rise. The capacity and cost of local computing infrastructures are likely to become prohibitive with the increase, especially as the breadth of questions continues to rise. Alternative computing architectures, in particular cloud computing environments, may help alleviate this increasing pressure and enable fast, large-scale, and cost-effective comparative genomics strategies going forward. To test this, we redesigned a typical comparative genomics algorithm, the reciprocal smallest distance algorithm (RSD), to run within Amazon's Elastic Computing Cloud (EC2). We then employed the RSD-cloud for ortholog calculations across a wide selection of fully sequenced genomes. We ran more than 300,000 RSD-cloud processes within the EC2. These jobs were farmed simultaneously to 100 high capacity compute nodes using the Amazon Web Service Elastic Map Reduce and included a wide mix of large and small genomes. The total computation time took just under 70 hours and cost a total of $6,302 USD. The effort to transform existing comparative genomics algorithms from local compute infrastructures is not trivial. However, the speed and flexibility of cloud computing environments provides a substantial boost with manageable cost. The procedure designed to transform the RSD algorithm into a cloud-ready application is readily adaptable to similar comparative genomics problems.

Lidar-based individual tree species classification using convolutional neural network

NASA Astrophysics Data System (ADS)

Mizoguchi, Tomohiro; Ishii, Akira; Nakamura, Hiroyuki; Inoue, Tsuyoshi; Takamatsu, Hisashi

2017-06-01

Terrestrial lidar is commonly used for detailed documentation in the field of forest inventory investigation. Recent improvements of point cloud processing techniques enabled efficient and precise computation of an individual tree shape parameters, such as breast-height diameter, height, and volume. However, tree species are manually specified by skilled workers to date. Previous works for automatic tree species classification mainly focused on aerial or satellite images, and few works have been reported for classification techniques using ground-based sensor data. Several candidate sensors can be considered for classification, such as RGB or multi/hyper spectral cameras. Above all candidates, we use terrestrial lidar because it can obtain high resolution point cloud in the dark forest. We selected bark texture for the classification criteria, since they clearly represent unique characteristics of each tree and do not change their appearance under seasonable variation and aged deterioration. In this paper, we propose a new method for automatic individual tree species classification based on terrestrial lidar using Convolutional Neural Network (CNN). The key component is the creation step of a depth image which well describe the characteristics of each species from a point cloud. We focus on Japanese cedar and cypress which cover the large part of domestic forest. Our experimental results demonstrate the effectiveness of our proposed method.

Development of the Landsat Data Continuity Mission Cloud Cover Assessment Algorithms

USGS Publications Warehouse

Scaramuzza, Pat; Bouchard, M.A.; Dwyer, John L.

2012-01-01

The upcoming launch of the Operational Land Imager (OLI) will start the next era of the Landsat program. However, the Automated Cloud-Cover Assessment (CCA) (ACCA) algorithm used on Landsat 7 requires a thermal band and is thus not suited for OLI. There will be a thermal instrument on the Landsat Data Continuity Mission (LDCM)-the Thermal Infrared Sensor-which may not be available during all OLI collections. This illustrates a need for CCA for LDCM in the absence of thermal data. To research possibilities for full-resolution OLI cloud assessment, a global data set of 207 Landsat 7 scenes with manually generated cloud masks was created. It was used to evaluate the ACCA algorithm, showing that the algorithm correctly classified 79.9% of a standard test subset of 3.95 109 pixels. The data set was also used to develop and validate two successor algorithms for use with OLI data-one derived from an off-the-shelf machine learning package and one based on ACCA but enhanced by a simple neural network. These comprehensive CCA algorithms were shown to correctly classify pixels as cloudy or clear 88.5% and 89.7% of the time, respectively.

Enhanced K-means clustering with encryption on cloud

NASA Astrophysics Data System (ADS)

Singh, Iqjot; Dwivedi, Prerna; Gupta, Taru; Shynu, P. G.

2017-11-01

This paper tries to solve the problem of storing and managing big files over cloud by implementing hashing on Hadoop in big-data and ensure security while uploading and downloading files. Cloud computing is a term that emphasis on sharing data and facilitates to share infrastructure and resources.[10] Hadoop is an open source software that gives us access to store and manage big files according to our needs on cloud. K-means clustering algorithm is an algorithm used to calculate distance between the centroid of the cluster and the data points. Hashing is a algorithm in which we are storing and retrieving data with hash keys. The hashing algorithm is called as hash function which is used to portray the original data and later to fetch the data stored at the specific key. [17] Encryption is a process to transform electronic data into non readable form known as cipher text. Decryption is the opposite process of encryption, it transforms the cipher text into plain text that the end user can read and understand well. For encryption and decryption we are using Symmetric key cryptographic algorithm. In symmetric key cryptography are using DES algorithm for a secure storage of the files. [3

Physical modeling of 3D and 4D laser imaging

NASA Astrophysics Data System (ADS)

Anna, Guillaume; Hamoir, Dominique; Hespel, Laurent; Lafay, Fabien; Rivière, Nicolas; Tanguy, Bernard

2010-04-01

Laser imaging offers potential for observation, for 3D terrain-mapping and classification as well as for target identification, including behind vegetation, camouflage or glass windows, at day and night, and under all-weather conditions. First generation systems deliver 3D point clouds. The threshold detection is largely affected by the local opto-geometric characteristics of the objects, leading to inaccuracies in the distances measured, and by partial occultation, leading to multiple echos. Second generation systems circumvent these limitations by recording the temporal waveforms received by the system, so that data processing can improve the telemetry and the point cloud better match the reality. Future algorithms may exploit the full potential of the 4D full-waveform data. Hence, being able to simulate point-cloud (3D) and full-waveform (4D) laser imaging is key. We have developped a numerical model for predicting the output data of 3D or 4D laser imagers. The model does account for the temporal and transverse characteristics of the laser pulse (i.e. of the "laser bullet") emitted by the system, its propagation through turbulent and scattering atmosphere, its interaction with the objects present in the field of view, and the characteristics of the optoelectronic reception path of the system.

Hierarchical trie packet classification algorithm based on expectation-maximization clustering

PubMed Central

Bi, Xia-an; Zhao, Junxia

2017-01-01

With the development of computer network bandwidth, packet classification algorithms which are able to deal with large-scale rule sets are in urgent need. Among the existing algorithms, researches on packet classification algorithms based on hierarchical trie have become an important packet classification research branch because of their widely practical use. Although hierarchical trie is beneficial to save large storage space, it has several shortcomings such as the existence of backtracking and empty nodes. This paper proposes a new packet classification algorithm, Hierarchical Trie Algorithm Based on Expectation-Maximization Clustering (HTEMC). Firstly, this paper uses the formalization method to deal with the packet classification problem by means of mapping the rules and data packets into a two-dimensional space. Secondly, this paper uses expectation-maximization algorithm to cluster the rules based on their aggregate characteristics, and thereby diversified clusters are formed. Thirdly, this paper proposes a hierarchical trie based on the results of expectation-maximization clustering. Finally, this paper respectively conducts simulation experiments and real-environment experiments to compare the performances of our algorithm with other typical algorithms, and analyzes the results of the experiments. The hierarchical trie structure in our algorithm not only adopts trie path compression to eliminate backtracking, but also solves the problem of low efficiency of trie updates, which greatly improves the performance of the algorithm. PMID:28704476

The generalization ability of online SVM classification based on Markov sampling.

PubMed

Xu, Jie; Yan Tang, Yuan; Zou, Bin; Xu, Zongben; Li, Luoqing; Lu, Yang

2015-03-01

In this paper, we consider online support vector machine (SVM) classification learning algorithms with uniformly ergodic Markov chain (u.e.M.c.) samples. We establish the bound on the misclassification error of an online SVM classification algorithm with u.e.M.c. samples based on reproducing kernel Hilbert spaces and obtain a satisfactory convergence rate. We also introduce a novel online SVM classification algorithm based on Markov sampling, and present the numerical studies on the learning ability of online SVM classification based on Markov sampling for benchmark repository. The numerical studies show that the learning performance of the online SVM classification algorithm based on Markov sampling is better than that of classical online SVM classification based on random sampling as the size of training samples is larger.

a Global Registration Algorithm of the Single-Closed Ring Multi-Stations Point Cloud

NASA Astrophysics Data System (ADS)

Yang, R.; Pan, L.; Xiang, Z.; Zeng, H.

2018-04-01

Aimed at the global registration problem of the single-closed ring multi-stations point cloud, a formula in order to calculate the error of rotation matrix was constructed according to the definition of error. The global registration algorithm of multi-station point cloud was derived to minimize the error of rotation matrix. And fast-computing formulas of transformation matrix with whose implementation steps and simulation experiment scheme was given. Compared three different processing schemes of multi-station point cloud, the experimental results showed that the effectiveness of the new global registration method was verified, and it could effectively complete the global registration of point cloud.

An approximate dynamic programming approach to resource management in multi-cloud scenarios

NASA Astrophysics Data System (ADS)

Pietrabissa, Antonio; Priscoli, Francesco Delli; Di Giorgio, Alessandro; Giuseppi, Alessandro; Panfili, Martina; Suraci, Vincenzo

2017-03-01

The programmability and the virtualisation of network resources are crucial to deploy scalable Information and Communications Technology (ICT) services. The increasing demand of cloud services, mainly devoted to the storage and computing, requires a new functional element, the Cloud Management Broker (CMB), aimed at managing multiple cloud resources to meet the customers' requirements and, simultaneously, to optimise their usage. This paper proposes a multi-cloud resource allocation algorithm that manages the resource requests with the aim of maximising the CMB revenue over time. The algorithm is based on Markov decision process modelling and relies on reinforcement learning techniques to find online an approximate solution.

Cloud Base Height Measurements at Manila Observatory: Initial Results from Constructed Paired Sky Imaging Cameras

NASA Astrophysics Data System (ADS)

Lagrosas, N.; Tan, F.; Antioquia, C. T.

2014-12-01

Fabricated all sky imagers are efficient and cost effective instruments for cloud detection and classification. Continuous operation of this instrument can result in the determination of cloud occurrence and cloud base heights for the paired system. In this study, a fabricated paired sky imaging system - consisting two commercial digital cameras (Canon Powershot A2300) enclosed in weatherproof containers - is developed in Manila Observatory for the purpose of determining cloud base heights at the Manila Observatory area. One of the cameras is placed on the rooftop of Manila Observatory and the other is placed on the rooftop of the university dormitory, 489m from the first camera. The cameras are programmed to simultaneously gather pictures every 5 min. Continuous operation of these cameras were implemented since the end of May of 2014 but data collection started end of October 2013. The data were processed following the algorithm proposed by Kassianov et al (2005). The processing involves the calculation of the merit function that determines the area of overlap of the two pictures. When two pictures are overlapped, the minimum of the merit function corresponds to the pixel column positions where the pictures have the best overlap. In this study, pictures of overcast sky prove to be difficult to process for cloud base height and were excluded from processing. The figure below shows the initial results of the hourly average of cloud base heights from data collected from November 2013 to July 2014. Measured cloud base heights ranged from 250m to 1.5km. These are the heights of cumulus and nimbus clouds that are dominant in this part of the world. Cloud base heights are low in the early hours of the day indicating low convection process during these times. However, the increase in the convection process in the atmosphere can be deduced from higher cloud base heights in the afternoon. The decrease of cloud base heights after 15:00 follows the trend of decreasing solar energy in the atmosphere after this time. The results show the potential of these instruments to determine cloud base heights on prolonged time intervals. The continuous operation of these instruments is implemented to gather seasonal variation of cloud base heights in this part of the world and to add to the much-needed dataset for future climate studies in Manila Observatory.

Object Based Image Analysis Combining High Spatial Resolution Imagery and Laser Point Clouds for Urban Land Cover

NASA Astrophysics Data System (ADS)

Zou, Xiaoliang; Zhao, Guihua; Li, Jonathan; Yang, Yuanxi; Fang, Yong

2016-06-01

With the rapid developments of the sensor technology, high spatial resolution imagery and airborne Lidar point clouds can be captured nowadays, which make classification, extraction, evaluation and analysis of a broad range of object features available. High resolution imagery, Lidar dataset and parcel map can be widely used for classification as information carriers. Therefore, refinement of objects classification is made possible for the urban land cover. The paper presents an approach to object based image analysis (OBIA) combing high spatial resolution imagery and airborne Lidar point clouds. The advanced workflow for urban land cover is designed with four components. Firstly, colour-infrared TrueOrtho photo and laser point clouds were pre-processed to derive the parcel map of water bodies and nDSM respectively. Secondly, image objects are created via multi-resolution image segmentation integrating scale parameter, the colour and shape properties with compactness criterion. Image can be subdivided into separate object regions. Thirdly, image objects classification is performed on the basis of segmentation and a rule set of knowledge decision tree. These objects imagery are classified into six classes such as water bodies, low vegetation/grass, tree, low building, high building and road. Finally, in order to assess the validity of the classification results for six classes, accuracy assessment is performed through comparing randomly distributed reference points of TrueOrtho imagery with the classification results, forming the confusion matrix and calculating overall accuracy and Kappa coefficient. The study area focuses on test site Vaihingen/Enz and a patch of test datasets comes from the benchmark of ISPRS WG III/4 test project. The classification results show higher overall accuracy for most types of urban land cover. Overall accuracy is 89.5% and Kappa coefficient equals to 0.865. The OBIA approach provides an effective and convenient way to combine high resolution imagery and Lidar ancillary data for classification of urban land cover.

Robust spike classification based on frequency domain neural waveform features.

PubMed

Yang, Chenhui; Yuan, Yuan; Si, Jennie

2013-12-01

We introduce a new spike classification algorithm based on frequency domain features of the spike snippets. The goal for the algorithm is to provide high classification accuracy, low false misclassification, ease of implementation, robustness to signal degradation, and objectivity in classification outcomes. In this paper, we propose a spike classification algorithm based on frequency domain features (CFDF). It makes use of frequency domain contents of the recorded neural waveforms for spike classification. The self-organizing map (SOM) is used as a tool to determine the cluster number intuitively and directly by viewing the SOM output map. After that, spike classification can be easily performed using clustering algorithms such as the k-Means. In conjunction with our previously developed multiscale correlation of wavelet coefficient (MCWC) spike detection algorithm, we show that the MCWC and CFDF detection and classification system is robust when tested on several sets of artificial and real neural waveforms. The CFDF is comparable to or outperforms some popular automatic spike classification algorithms with artificial and real neural data. The detection and classification of neural action potentials or neural spikes is an important step in single-unit-based neuroscientific studies and applications. After the detection of neural snippets potentially containing neural spikes, a robust classification algorithm is applied for the analysis of the snippets to (1) extract similar waveforms into one class for them to be considered coming from one unit, and to (2) remove noise snippets if they do not contain any features of an action potential. Usually, a snippet is a small 2 or 3 ms segment of the recorded waveform, and differences in neural action potentials can be subtle from one unit to another. Therefore, a robust, high performance classification system like the CFDF is necessary. In addition, the proposed algorithm does not require any assumptions on statistical properties of the noise and proves to be robust under noise contamination.

Voice based gender classification using machine learning

NASA Astrophysics Data System (ADS)

Raahul, A.; Sapthagiri, R.; Pankaj, K.; Vijayarajan, V.

2017-11-01

Gender identification is one of the major problem speech analysis today. Tracing the gender from acoustic data i.e., pitch, median, frequency etc. Machine learning gives promising results for classification problem in all the research domains. There are several performance metrics to evaluate algorithms of an area. Our Comparative model algorithm for evaluating 5 different machine learning algorithms based on eight different metrics in gender classification from acoustic data. Agenda is to identify gender, with five different algorithms: Linear Discriminant Analysis (LDA), K-Nearest Neighbour (KNN), Classification and Regression Trees (CART), Random Forest (RF), and Support Vector Machine (SVM) on basis of eight different metrics. The main parameter in evaluating any algorithms is its performance. Misclassification rate must be less in classification problems, which says that the accuracy rate must be high. Location and gender of the person have become very crucial in economic markets in the form of AdSense. Here with this comparative model algorithm, we are trying to assess the different ML algorithms and find the best fit for gender classification of acoustic data.

Tracing Forest Change through 40 Years on Two Continents with the BULC Algorithm and Google Earth Engine

NASA Astrophysics Data System (ADS)

Cardille, J. A.; Crowley, M.; Fortin, J. A.; Lee, J.; Perez, E.; Sleeter, B. M.; Thau, D.

2016-12-01

With the opening of the Landsat archive, researchers have a vast new data source teeming with imagery and potential. Beyond Landsat, data from other sensors is newly available as well: these include ALOS/PALSAR, Sentinel-1 and -2, MERIS, and many more. Google Earth Engine, developed to organize and provide analysis tools for these immense data sets, is an ideal platform for researchers trying to sift through huge image stacks. It offers nearly unlimited processing power and storage with a straightforward programming interface. Yet labeling land-cover change through time remains challenging given the current state of the art for interpreting remote sensing image sequences. Moreover, combining data from very different image platforms remains quite difficult. To address these challenges, we developed the BULC algorithm (Bayesian Updating of Land Cover), designed for the continuous updating of land-cover classifications through time in large data sets. The algorithm ingests data from any of the wide variety of earth-resources sensors; it maintains a running estimate of land-cover probabilities and the most probable class at all time points along a sequence of events. Here we compare BULC results from two study sites that witnessed considerable forest change in the last 40 years: the Pacific Northwest of the United States and the Mato Grosso region of Brazil. In Brazil, we incorporated rough classifications from more than 100 images of varying quality, mixing imagery from more than 10 different sensors. In the Pacific Northwest, we used BULC to identify forest changes due to logging and urbanization from 1973 to the present. Both regions had classification sequences that were better than many of the component days, effectively ignoring clouds and other unwanted noise while fusing the information contained on several platforms. As we leave remote sensing's data-poor era and enter a period with multiple looks at Earth's surface from multiple sensors over a short period of time, the BULC algorithm can help to sift through images of varying quality in Google Earth Engine to extract the most useful information for mapping the state and history of Earth's land cover.

Tracing Forest Change through 40 Years on Two Continents with the BULC Algorithm and Google Earth Engine

NASA Astrophysics Data System (ADS)

Cardille, J. A.

2015-12-01

With the opening of the Landsat archive, researchers have a vast new data source teeming with imagery and potential. Beyond Landsat, data from other sensors is newly available as well: these include ALOS/PALSAR, Sentinel-1 and -2, MERIS, and many more. Google Earth Engine, developed to organize and provide analysis tools for these immense data sets, is an ideal platform for researchers trying to sift through huge image stacks. It offers nearly unlimited processing power and storage with a straightforward programming interface. Yet labeling forest change through time remains challenging given the current state of the art for interpreting remote sensing image sequences. Moreover, combining data from very different image platforms remains quite difficult. To address these challenges, we developed the BULC algorithm (Bayesian Updating of Land Cover), designed for the continuous updating of land-cover classifications through time in large data sets. The algorithm ingests data from any of the wide variety of earth-resources sensors; it maintains a running estimate of land-cover probabilities and the most probable class at all time points along a sequence of events. Here we compare BULC results from two study sites that witnessed considerable forest change in the last 40 years: the Pacific Northwest of the United States and the Mato Grosso region of Brazil. In Brazil, we incorporated rough classifications from more than 100 images of varying quality, mixing imagery from more than 10 different sensors. In the Pacific Northwest, we used BULC to identify forest changes due to logging and urbanization from 1973 to the present. Both regions had classification sequences that were better than many of the component days, effectively ignoring clouds and other unwanted signal while fusing the information contained on several platforms. As we leave remote sensing's data-poor era and enter a period with multiple looks at Earth's surface from multiple sensors over a short period of time, this algorithm may help to sift through images of varying quality in Google Earth Engine to extract the most useful information for mapping.

Adaptive Grouping Cloud Model Shuffled Frog Leaping Algorithm for Solving Continuous Optimization Problems

PubMed Central

Liu, Haorui; Yi, Fengyan; Yang, Heli

2016-01-01

The shuffled frog leaping algorithm (SFLA) easily falls into local optimum when it solves multioptimum function optimization problem, which impacts the accuracy and convergence speed. Therefore this paper presents grouped SFLA for solving continuous optimization problems combined with the excellent characteristics of cloud model transformation between qualitative and quantitative research. The algorithm divides the definition domain into several groups and gives each group a set of frogs. Frogs of each region search in their memeplex, and in the search process the algorithm uses the “elite strategy” to update the location information of existing elite frogs through cloud model algorithm. This method narrows the searching space and it can effectively improve the situation of a local optimum; thus convergence speed and accuracy can be significantly improved. The results of computer simulation confirm this conclusion. PMID:26819584

Load balancing prediction method of cloud storage based on analytic hierarchy process and hybrid hierarchical genetic algorithm.

PubMed

Zhou, Xiuze; Lin, Fan; Yang, Lvqing; Nie, Jing; Tan, Qian; Zeng, Wenhua; Zhang, Nian

2016-01-01

With the continuous expansion of the cloud computing platform scale and rapid growth of users and applications, how to efficiently use system resources to improve the overall performance of cloud computing has become a crucial issue. To address this issue, this paper proposes a method that uses an analytic hierarchy process group decision (AHPGD) to evaluate the load state of server nodes. Training was carried out by using a hybrid hierarchical genetic algorithm (HHGA) for optimizing a radial basis function neural network (RBFNN). The AHPGD makes the aggregative indicator of virtual machines in cloud, and become input parameters of predicted RBFNN. Also, this paper proposes a new dynamic load balancing scheduling algorithm combined with a weighted round-robin algorithm, which uses the predictive periodical load value of nodes based on AHPPGD and RBFNN optimized by HHGA, then calculates the corresponding weight values of nodes and makes constant updates. Meanwhile, it keeps the advantages and avoids the shortcomings of static weighted round-robin algorithm.

Validation of Local-Cloud Model Outputs With the GOES Satellite Imagery

NASA Astrophysics Data System (ADS)

Malek, E.

2005-05-01

Clouds (visible aggregations of minute droplets of water or tiny crystals of ice suspended in the air) affect the radiation budget of our planet by reflecting, absorbing and scattering solar radiation, and the re-emission of terrestrial radiation. They affect the weather and climate by positive or negative feedbacks. Many researchers have worked on the parameterization of clouds and their effects on the radiation budget. There is little information about ground-based approaches for continuous evaluation of cloud, such as cloud base height, cloud base temperature, and cloud coverage, at local and regional scales. This present article deals with the development of an algorithm for continuous (day and night) evaluation of cloud base temperature, cloud base height and percent of skies covered by cloud at local scale throughout the year. The Vaisala model CT-12K laser beam ceilometer is used at the Automated Surface Observing Systems (ASOS) to measure the cloud base height and report the sky conditions on an hourly basis or at shorter intervals. This laser ceilometer is a fixed-type whose transmitter and receiver point straight up at the cloud (if any) base. It is unable to measure clouds that are not above the sensor. To report cloudiness at the local scale, many of these type of ceilometers are needed. This is not a perfect method for cloud measurement. A single cloud hanging overhead the sensor will cause overcast readings, whereas, a hole in the clouds could cause a clear reading to be reported. To overcome this problem, we have set up a ventilated radiation station at Logan-Cache airport, Utah, U.S.A., since 1995, which is equipped with one of the above-mentioned ceilometers. This radiation station (composed of pyranometers, pyrgeometers and net radiometer) provides continuous measurements of incoming and outgoing shortwave and longwave radiation and the net radiation throughout the year. We have also measured the surface temperature and pressure, the 2-m air temperature and humidity, precipitation, and the 3-m wind and direction at this station. Having the air temperature, moisture, and the measured cloudless incoming longwave (atmospheric) radiation during 1999 through 2004, based upon the ASOS and the algorithm data, we found the appropriate formula (among four reported approaches) for computation of the cloudless-skies atmospheric emissivity. Considering the additional longwave radiation captured by the facing-up pyrgeometer during the cloudy skies, coming from the cloud in the wave band which the gaseous emission lacks (from 8-13 ìm), we developed an algorithm which provides the continuous 20-min cloud information (cloud base height, cloud base temperature, and percent of skies covered by cloud) over the Cache Valley during day and night throughout the year. The comparisons between the ASOS and the algorithm data during the period of 8-12 June, 2004 are reported in this article. The proposed algorithm is a promising approach for evaluation of the cloud base temperature, cloud base height, and percent of skies covered by cloud at the local scale throughout the year. It also reports the comparison between model outputs and GOES 10 satellite images.

A semi-supervised classification algorithm using the TAD-derived background as training data

NASA Astrophysics Data System (ADS)

Fan, Lei; Ambeau, Brittany; Messinger, David W.

2013-05-01

In general, spectral image classification algorithms fall into one of two categories: supervised and unsupervised. In unsupervised approaches, the algorithm automatically identifies clusters in the data without a priori information about those clusters (except perhaps the expected number of them). Supervised approaches require an analyst to identify training data to learn the characteristics of the clusters such that they can then classify all other pixels into one of the pre-defined groups. The classification algorithm presented here is a semi-supervised approach based on the Topological Anomaly Detection (TAD) algorithm. The TAD algorithm defines background components based on a mutual k-Nearest Neighbor graph model of the data, along with a spectral connected components analysis. Here, the largest components produced by TAD are used as regions of interest (ROI's),or training data for a supervised classification scheme. By combining those ROI's with a Gaussian Maximum Likelihood (GML) or a Minimum Distance to the Mean (MDM) algorithm, we are able to achieve a semi supervised classification method. We test this classification algorithm against data collected by the HyMAP sensor over the Cooke City, MT area and University of Pavia scene.

Fast and Robust Segmentation and Classification for Change Detection in Urban Point Clouds

NASA Astrophysics Data System (ADS)

Roynard, X.; Deschaud, J.-E.; Goulette, F.

2016-06-01

Change detection is an important issue in city monitoring to analyse street furniture, road works, car parking, etc. For example, parking surveys are needed but are currently a laborious task involving sending operators in the streets to identify the changes in car locations. In this paper, we propose a method that performs a fast and robust segmentation and classification of urban point clouds, that can be used for change detection. We apply this method to detect the cars, as a particular object class, in order to perform parking surveys automatically. A recently proposed method already addresses the need for fast segmentation and classification of urban point clouds, using elevation images. The interest to work on images is that processing is much faster, proven and robust. However there may be a loss of information in complex 3D cases: for example when objects are one above the other, typically a car under a tree or a pedestrian under a balcony. In this paper we propose a method that retain the three-dimensional information while preserving fast computation times and improving segmentation and classification accuracy. It is based on fast region-growing using an octree, for the segmentation, and specific descriptors with Random-Forest for the classification. Experiments have been performed on large urban point clouds acquired by Mobile Laser Scanning. They show that the method is as fast as the state of the art, and that it gives more robust results in the complex 3D cases.

New optical package and algorithms for accurate estimation and interactive recording of the cloud cover information over land and sea

NASA Astrophysics Data System (ADS)

Krinitskiy, Mikhail; Sinitsyn, Alexey; Gulev, Sergey

2014-05-01

Cloud fraction is a critical parameter for the accurate estimation of short-wave and long-wave radiation - one of the most important surface fluxes over sea and land. Massive estimates of the total cloud cover as well as cloud amount for different layers of clouds are available from visual observations, satellite measurements and reanalyses. However, these data are subject of different uncertainties and need continuous validation against highly accurate in-situ measurements. Sky imaging with high resolution fish eye camera provides an excellent opportunity for collecting cloud cover data supplemented with additional characteristics hardly available from routine visual observations (e.g. structure of cloud cover under broken cloud conditions, parameters of distribution of cloud dimensions). We present operational automatic observational package which is based on fish eye camera taking sky images with high resolution (up to 1Hz) in time and a spatial resolution of 968x648px. This spatial resolution has been justified as an optimal by several sensitivity experiments. For the use of the package at research vessel when the horizontal positioning becomes critical, a special extension of the hardware and software to the package has been developed. These modules provide the explicit detection of the optimal moment for shooting. For the post processing of sky images we developed a software realizing the algorithm of the filtering of sunburn effect in case of small and moderate could cover and broken cloud conditions. The same algorithm accurately quantifies the cloud fraction by analyzing color mixture for each point and introducing the so-called "grayness rate index" for every pixel. The accuracy of the algorithm has been tested using the data collected during several campaigns in 2005-2011 in the North Atlantic Ocean. The collection of images included more than 3000 images for different cloud conditions supplied with observations of standard parameters. The system is fully autonomous and has a block for digital data collection at the hard disk. The system has been tested for a wide range of open ocean cloud conditions and we will demonstrate some pilot results of data processing and physical interpretation of fractional cloud cover estimation.

Modelled and measured effects of clouds on UV Aerosol Indices on a local, regional, and global scale

NASA Astrophysics Data System (ADS)

Penning de Vries, M.; Wagner, T.

2011-12-01

The UV Aerosol Indices (UVAI) form one of very few available tools in satellite remote sensing that provide information on aerosol absorption. The UVAI are also quite insensitive to surface type and are determined in the presence of clouds - situations where most aerosol retrieval algorithms do not work. The UVAI are most sensitive to elevated layers of absorbing aerosols, such as mineral dust and smoke, but they can also be used to study non-absorbing aerosols, such as sulphate and secondary organic aerosols. Although UVAI are determined for cloud-contaminated pixels, clouds do affect the value of UVAI in several ways: (1) they shield the underlying scene (potentially containing aerosols) from view, (2) they enhance the apparent surface albedo of an elevated aerosol layer, and (3) clouds unpolluted by aerosols also yield non-zero UVAI, here referred to as "cloudUVAI". The main purpose of this paper is to demonstrate that clouds can cause significant UVAI and that this cloudUVAI can be well modelled using simple assumptions on cloud properties. To this aim, we modelled cloudUVAI by using measured cloud optical parameters - either with low spatial resolution from SCIAMACHY, or high resolution from MERIS - as input. The modelled cloudUVAI were compared with UVAI determined from SCIAMACHY reflectances on different spatial (local, regional and global) and temporal scales (single measurement, daily means and seasonal means). The general dependencies of UVAI on cloud parameters were quite well reproduced, but several issues remain unclear: compared to the modelled cloudUVAI, measured UVAI show a bias, in particular for large cloud fractions. Also, the spread in measured UVAI is larger than in modelled cloudUVAI. In addition to the original, Lambert Equivalent Reflector (LER)-based UVAI algorithm, we have also investigated the effects of clouds on UVAI determined using the so-called Modified LER (MLER) algorithm (currently applied to TOMS and OMI data). For medium-sized clouds the MLER algorithm performs better (UVAI are closer to 0), but like for LER UVAI, MLER UVAI can become as large as -1.2 for small clouds and deviate significantly from zero for cloud fractions near 1. The effects of clouds should therefore also be taken into account when MLER UVAI data are used. Because the effects of clouds and aerosols on UVAI are not independent, a simple subtraction of modelled cloudUVAI from measured UVAI does not yield a UVAI representative of a cloud-free scene when aerosols are present. We here propose a first, simple approach for the correction of cloud effects on UVAI. The method is shown to work reasonably well for small to medium-sized clouds located above aerosols.

Correction of Rayleigh Scattering Effects in Cloud Optical Thickness Retrievals

NASA Technical Reports Server (NTRS)

Wang, Meng-Hua; King, Michael D.

1997-01-01

We present results that demonstrate the effects of Rayleigh scattering on the 9 retrieval of cloud optical thickness at a visible wavelength (0.66 Am). The sensor-measured radiance at a visible wavelength (0.66 Am) is usually used to infer remotely the cloud optical thickness from aircraft or satellite instruments. For example, we find that without removing Rayleigh scattering effects, errors in the retrieved cloud optical thickness for a thin water cloud layer (T = 2.0) range from 15 to 60%, depending on solar zenith angle and viewing geometry. For an optically thick cloud (T = 10), on the other hand, errors can range from 10 to 60% for large solar zenith angles (0-60 deg) because of enhanced Rayleigh scattering. It is therefore particularly important to correct for Rayleigh scattering contributions to the reflected signal from a cloud layer both (1) for the case of thin clouds and (2) for large solar zenith angles and all clouds. On the basis of the single scattering approximation, we propose an iterative method for effectively removing Rayleigh scattering contributions from the measured radiance signal in cloud optical thickness retrievals. The proposed correction algorithm works very well and can easily be incorporated into any cloud retrieval algorithm. The Rayleigh correction method is applicable to cloud at any pressure, providing that the cloud top pressure is known to within +/- 100 bPa. With the Rayleigh correction the errors in retrieved cloud optical thickness are usually reduced to within 3%. In cases of both thin cloud layers and thick ,clouds with large solar zenith angles, the errors are usually reduced by a factor of about 2 to over 10. The Rayleigh correction algorithm has been tested with simulations for realistic cloud optical and microphysical properties with different solar and viewing geometries. We apply the Rayleigh correction algorithm to the cloud optical thickness retrievals from experimental data obtained during the Atlantic Stratocumulus Transition Experiment (ASTEX) conducted near the Azores in June 1992 and compare these results to corresponding retrievals obtained using 0.88 Am. These results provide an example of the Rayleigh scattering effects on thin clouds and further test the Rayleigh correction scheme. Using a nonabsorbing near-infrared wavelength lambda (0.88 Am) in retrieving cloud optical thickness is only applicable over oceans, however, since most land surfaces are highly reflective at 0.88 Am. Hence successful global retrievals of cloud optical thickness should remove Rayleigh scattering effects when using reflectance measurements at 0.66 Am.

Bias adjustment of infrared-based rainfall estimation using Passive Microwave satellite rainfall data

NASA Astrophysics Data System (ADS)

Karbalaee, Negar; Hsu, Kuolin; Sorooshian, Soroosh; Braithwaite, Dan

2017-04-01

This study explores using Passive Microwave (PMW) rainfall estimation for spatial and temporal adjustment of Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks-Cloud Classification System (PERSIANN-CCS). The PERSIANN-CCS algorithm collects information from infrared images to estimate rainfall. PERSIANN-CCS is one of the algorithms used in the Integrated Multisatellite Retrievals for GPM (Global Precipitation Mission) estimation for the time period PMW rainfall estimations are limited or not available. Continued improvement of PERSIANN-CCS will support Integrated Multisatellite Retrievals for GPM for current as well as retrospective estimations of global precipitation. This study takes advantage of the high spatial and temporal resolution of GEO-based PERSIANN-CCS estimation and the more effective, but lower sample frequency, PMW estimation. The Probability Matching Method (PMM) was used to adjust the rainfall distribution of GEO-based PERSIANN-CCS toward that of PMW rainfall estimation. The results show that a significant improvement of global PERSIANN-CCS rainfall estimation is obtained.

Regularised extreme learning machine with misclassification cost and rejection cost for gene expression data classification.

PubMed

Lu, Huijuan; Wei, Shasha; Zhou, Zili; Miao, Yanzi; Lu, Yi

2015-01-01

The main purpose of traditional classification algorithms on bioinformatics application is to acquire better classification accuracy. However, these algorithms cannot meet the requirement that minimises the average misclassification cost. In this paper, a new algorithm of cost-sensitive regularised extreme learning machine (CS-RELM) was proposed by using probability estimation and misclassification cost to reconstruct the classification results. By improving the classification accuracy of a group of small sample which higher misclassification cost, the new CS-RELM can minimise the classification cost. The 'rejection cost' was integrated into CS-RELM algorithm to further reduce the average misclassification cost. By using Colon Tumour dataset and SRBCT (Small Round Blue Cells Tumour) dataset, CS-RELM was compared with other cost-sensitive algorithms such as extreme learning machine (ELM), cost-sensitive extreme learning machine, regularised extreme learning machine, cost-sensitive support vector machine (SVM). The results of experiments show that CS-RELM with embedded rejection cost could reduce the average cost of misclassification and made more credible classification decision than others.

An AVHRR Cloud Classification Database Typed by Experts

DTIC Science & Technology

1993-10-01

analysis. Naval Research Laboratory, Monterey, CA. 110 pp. Gallaudet , Timothy C. and James J. Simpson, 1991: Automated cloud screening of AVHRR imagery...1987) and Saunders and Kriebel (1988a,b) have used threshold techniques to classify clouds. Gallaudet and Simpson (1991) have used split-and-merge

Long Term Cloud Property Datasets From MODIS and AVHRR Using the CERES Cloud Algorithm

NASA Technical Reports Server (NTRS)

Minnis, Patrick; Bedka, Kristopher M.; Doelling, David R.; Sun-Mack, Sunny; Yost, Christopher R.; Trepte, Qing Z.; Bedka, Sarah T.; Palikonda, Rabindra; Scarino, Benjamin R.; Chen, Yan;

Machine learning based cloud mask algorithm driven by radiative transfer modeling

NASA Astrophysics Data System (ADS)

Chen, N.; Li, W.; Tanikawa, T.; Hori, M.; Shimada, R.; Stamnes, K. H.

2017-12-01

Cloud detection is a critically important first step required to derive many satellite data products. Traditional threshold based cloud mask algorithms require a complicated design process and fine tuning for each sensor, and have difficulty over snow/ice covered areas. With the advance of computational power and machine learning techniques, we have developed a new algorithm based on a neural network classifier driven by extensive radiative transfer modeling. Statistical validation results obtained by using collocated CALIOP and MODIS data show that its performance is consistent over different ecosystems and significantly better than the MODIS Cloud Mask (MOD35 C6) during the winter seasons over mid-latitude snow covered areas. Simulations using a reduced number of satellite channels also show satisfactory results, indicating its flexibility to be configured for different sensors.

A spectrum fractal feature classification algorithm for agriculture crops with hyper spectrum image

NASA Astrophysics Data System (ADS)

Su, Junying

2011-11-01

A fractal dimension feature analysis method in spectrum domain for hyper spectrum image is proposed for agriculture crops classification. Firstly, a fractal dimension calculation algorithm in spectrum domain is presented together with the fast fractal dimension value calculation algorithm using the step measurement method. Secondly, the hyper spectrum image classification algorithm and flowchart is presented based on fractal dimension feature analysis in spectrum domain. Finally, the experiment result of the agricultural crops classification with FCL1 hyper spectrum image set with the proposed method and SAM (spectral angle mapper). The experiment results show it can obtain better classification result than the traditional SAM feature analysis which can fulfill use the spectrum information of hyper spectrum image to realize precision agricultural crops classification.

Image-classification-based global dimming algorithm for LED backlights in LCDs

NASA Astrophysics Data System (ADS)

Qibin, Feng; Huijie, He; Dong, Han; Lei, Zhang; Guoqiang, Lv

2015-07-01

Backlight dimming can help LCDs reduce power consumption and improve CR. With fixed parameters, dimming algorithm cannot achieve satisfied effects for all kinds of images. The paper introduces an image-classification-based global dimming algorithm. The proposed classification method especially for backlight dimming is based on luminance and CR of input images. The parameters for backlight dimming level and pixel compensation are adaptive with image classifications. The simulation results show that the classification based dimming algorithm presents 86.13% power reduction improvement compared with dimming without classification, with almost same display quality. The prototype is developed. There are no perceived distortions when playing videos. The practical average power reduction of the prototype TV is 18.72%, compared with common TV without dimming.

A new algorithm combining geostatistics with the surrogate data approach to increase the accuracy of comparisons of point radiation measurements with cloud measurements

NASA Astrophysics Data System (ADS)

Venema, V. K. C.; Lindau, R.; Varnai, T.; Simmer, C.

2009-04-01

Two main groups of statistical methods used in the Earth sciences are geostatistics and stochastic modelling. Geostatistical methods, such as various kriging algorithms, aim at estimating the mean value for every point as well as possible. In case of sparse measurements, such fields have less variability at small scales and a narrower distribution as the true field. This can lead to biases if a nonlinear process is simulated on such a kriged field. Stochastic modelling aims at reproducing the structure of the data. One of the stochastic modelling methods, the so-called surrogate data approach, replicates the value distribution and power spectrum of a certain data set. However, while stochastic methods reproduce the statistical properties of the data, the location of the measurement is not considered. Because radiative transfer through clouds is a highly nonlinear process it is essential to model the distribution (e.g. of optical depth, extinction, liquid water content or liquid water path) accurately as well as the correlations in the cloud field because of horizontal photon transport. This explains the success of surrogate cloud fields for use in 3D radiative transfer studies. However, up to now we could only achieve good results for the radiative properties averaged over the field, but not for a radiation measurement located at a certain position. Therefore we have developed a new algorithm that combines the accuracy of stochastic (surrogate) modelling with the positioning capabilities of kriging. In this way, we can automatically profit from the large geostatistical literature and software. The algorithm is tested on cloud fields from large eddy simulations (LES). On these clouds a measurement is simulated. From the pseudo-measurement we estimated the distribution and power spectrum. Furthermore, the pseudo-measurement is kriged to a field the size of the final surrogate cloud. The distribution, spectrum and the kriged field are the inputs to the algorithm. This algorithm is similar to the standard iterative amplitude adjusted Fourier transform (IAAFT) algorithm, but has an additional iterative step in which the surrogate field is nudged towards the kriged field. The nudging strength is gradually reduced to zero. We work with four types of pseudo-measurements: one zenith pointing measurement (which together with the wind produces a line measurement), five zenith pointing measurements, a slow and a fast azimuth scan (which together with the wind produce spirals). Because we work with LES clouds and the truth is known, we can validate the algorithm by performing 3D radiative transfer calculations on the original LES clouds and on the new surrogate clouds. For comparison also the radiative properties of the kriged fields and standard surrogate fields are computed. Preliminary results already show that these new surrogate clouds reproduce the structure of the original clouds very well and the minima and maxima are located where the pseudo-measurements sees them. The main limitation seems to be the amount of data, which is especially very limited in case of just one zenith pointing measurement.

Sentinel-1 Archive and Processing in the Cloud using the Hybrid Pluggable Processing Pipeline (HyP3) at the ASF DAAC

NASA Astrophysics Data System (ADS)

Arko, S. A.; Hogenson, R.; Geiger, A.; Herrmann, J.; Buechler, B.; Hogenson, K.

2016-12-01

In the coming years there will be an unprecedented amount of SAR data available on a free and open basis to research and operational users around the globe. The Alaska Satellite Facility (ASF) DAAC hosts, through an international agreement, data from the Sentinel-1 spacecraft and will be hosting data from the upcoming NASA ISRO SAR (NISAR) mission. To more effectively manage and exploit these vast datasets, ASF DAAC has begun moving portions of the archive to the cloud and utilizing cloud services to provide higher-level processing on the data. The Hybrid Pluggable Processing Pipeline (HyP3) project is designed to support higher-level data processing in the cloud and extend the capabilities of researchers to larger scales. Built upon a set of core Amazon cloud services, the HyP3 system allows users to request data processing using a number of canned algorithms or their own algorithms once they have been uploaded to the cloud. The HyP3 system automatically accesses the ASF cloud-based archive through the DAAC RESTful application programming interface and processes the data on Amazon's elastic compute cluster (EC2). Final products are distributed through Amazon's simple storage service (S3) and are available for user download. This presentation will provide an overview of ASF DAAC's activities moving the Sentinel-1 archive into the cloud and developing the integrated HyP3 system, covering both the benefits and difficulties of working in the cloud. Additionally, we will focus on the utilization of HyP3 for higher-level processing of SAR data. Two example algorithms, for sea-ice tracking and change detection, will be discussed as well as the mechanism for integrating new algorithms into the pipeline for community use.

Clouds and Climate Change. Understanding Global Change: Earth Science and Human Impacts. Global Change Instruction Program.

ERIC Educational Resources Information Center

Shaw, Glenn E.

The Global Change Instruction Program was designed by college professors to fill a need for interdisciplinary materials on the emerging science of global change. This instructional module introduces the basic features and classifications of clouds and cloud cover, and explains how clouds form, what they are made of, what roles they play in…

Atrial Fibrillation Screening in Nonmetropolitan Areas Using a Telehealth Surveillance System With an Embedded Cloud-Computing Algorithm: Prospective Pilot Study.

PubMed

Chen, Ying-Hsien; Hung, Chi-Sheng; Huang, Ching-Chang; Hung, Yu-Chien; Hwang, Juey-Jen; Ho, Yi-Lwun

2017-09-26

Atrial fibrillation (AF) is a common form of arrhythmia that is associated with increased risk of stroke and mortality. Detecting AF before the first complication occurs is a recognized priority. No previous studies have examined the feasibility of undertaking AF screening using a telehealth surveillance system with an embedded cloud-computing algorithm; we address this issue in this study. The objective of this study was to evaluate the feasibility of AF screening in nonmetropolitan areas using a telehealth surveillance system with an embedded cloud-computing algorithm. We conducted a prospective AF screening study in a nonmetropolitan area using a single-lead electrocardiogram (ECG) recorder. All ECG measurements were reviewed on the telehealth surveillance system and interpreted by the cloud-computing algorithm and a cardiologist. The process of AF screening was evaluated with a satisfaction questionnaire. Between March 11, 2016 and August 31, 2016, 967 ECGs were recorded from 922 residents in nonmetropolitan areas. A total of 22 (2.4%, 22/922) residents with AF were identified by the physician's ECG interpretation, and only 0.2% (2/967) of ECGs contained significant artifacts. The novel cloud-computing algorithm for AF detection had a sensitivity of 95.5% (95% CI 77.2%-99.9%) and specificity of 97.7% (95% CI 96.5%-98.5%). The overall satisfaction score for the process of AF screening was 92.1%. AF screening in nonmetropolitan areas using a telehealth surveillance system with an embedded cloud-computing algorithm is feasible. ©Ying-Hsien Chen, Chi-Sheng Hung, Ching-Chang Huang, Yu-Chien Hung, Juey-Jen Hwang, Yi-Lwun Ho. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 26.09.2017.

Patient identification using a near-infrared laser scanner

NASA Astrophysics Data System (ADS)

Manit, Jirapong; Bremer, Christina; Schweikard, Achim; Ernst, Floris

2017-03-01

We propose a new biometric approach where the tissue thickness of a person's forehead is used as a biometric feature. Given that the spatial registration of two 3D laser scans of the same human face usually produces a low error value, the principle of point cloud registration and its error metric can be applied to human classification techniques. However, by only considering the spatial error, it is not possible to reliably verify a person's identity. We propose to use a novel near-infrared laser-based head tracking system to determine an additional feature, the tissue thickness, and include this in the error metric. Using MRI as a ground truth, data from the foreheads of 30 subjects was collected from which a 4D reference point cloud was created for each subject. The measurements from the near-infrared system were registered with all reference point clouds using the ICP algorithm. Afterwards, the spatial and tissue thickness errors were extracted, forming a 2D feature space. For all subjects, the lowest feature distance resulted from the registration of a measurement and the reference point cloud of the same person. The combined registration error features yielded two clusters in the feature space, one from the same subject and another from the other subjects. When only the tissue thickness error was considered, these clusters were less distinct but still present. These findings could help to raise safety standards for head and neck cancer patients and lays the foundation for a future human identification technique.

Comparison of Cloud and Aerosol Detection between CERES Edition 3 Cloud Mask and CALIPSO Version 2 Data Products

NASA Astrophysics Data System (ADS)

Trepte, Qing; Minnis, Patrick; Sun-Mack, Sunny; Trepte, Charles

Clouds and aerosol play important roles in the global climate system. Accurately detecting their presence, altitude, and properties using satellite radiance measurements is a crucial first step in determining their influence on surface and top-of-atmosphere radiative fluxes. This paper presents a comparison analysis of a new version of the Clouds and Earth's Radiant Energy System (CERES) Edition 3 cloud detection algorithms using Aqua MODIS data with the recently released Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) Version 2 Vertical Feature Mask (VFM). Improvements in CERES Edition 3 cloud mask include dust detection, thin cirrus tests, enhanced low cloud detection at night, and a smoother transition from mid-latitude to polar regions. For the CALIPSO Version 2 data set, changes to the lidar calibration can result in significant improvements to its identification of optically thick aerosol layers. The Aqua and CALIPSO satellites, part of the A-train satellite constellation, provide a unique opportunity for validating passive sensor cloud and aerosol detection using an active sensor. In this paper, individual comparison cases will be discussed for different types of clouds and aerosols over various surfaces, for daytime and nighttime conditions, and for regions ranging from the tropics to the poles. Examples will include an assessment of the CERES detection algorithm for optically thin cirrus, marine stratus, and polar night clouds as well as its ability to characterize Saharan dust plumes off the African coast. With the CALIPSO lidar's unique ability to probe the vertical structure of clouds and aerosol layers, it provides an excellent validation data set for cloud detection algorithms, especially for polar nighttime clouds.

Examination of Regional Trends in Cloud Properties over Surface Sites Derived from MODIS and AVHRR using the CERES Cloud Algorithm

NASA Astrophysics Data System (ADS)

Smith, W. L., Jr.; Minnis, P.; Bedka, K. M.; Sun-Mack, S.; Chen, Y.; Doelling, D. R.; Kato, S.; Rutan, D. A.

2017-12-01

Recent studies analyzing long-term measurements of surface insolation at ground sites suggest that decadal-scale trends of increasing (brightening) and decreasing (dimming) downward solar flux have occurred at various times over the last century. Regional variations have been reported that range from near 0 Wm-2/decade to as large as 9 Wm-2/decade depending on the location and time period analyzed. The more significant trends have been attributed to changes in overhead clouds and aerosols, although quantifying their relative impacts using independent observations has been difficult, owing in part to a lack of consistent long-term measurements of cloud properties. This paper examines new satellite based records of cloud properties derived from MODIS (2000-present) and AVHRR (1981- present) data to infer cloud property trends over a number of surface radiation sites across the globe. The MODIS cloud algorithm was developed for the NASA Clouds and the Earth's Radiant Energy System (CERES) project to provide a consistent record of cloud properties to help improve broadband radiation measurements and to better understand cloud radiative effects. The CERES-MODIS cloud algorithm has been modified to analyze other satellites including the AVHRR on the NOAA satellites. Compared to MODIS, obtaining consistent cloud properties over a long period from AVHRR is a much more significant challenge owing to the number of different satellites, instrument calibration uncertainties, orbital drift and other factors. Nevertheless, both the MODIS and AVHRR cloud properties will be analyzed to determine trends, and their level of consistency and correspondence with surface radiation trends derived from the ground-based radiometer data. It is anticipated that this initial study will contribute to an improved understanding of surface solar radiation trends and their relationship to clouds.

Remote Sensing of the Radiative and Microphysical Properties of Clouds during TC4: Results from MAS, MASTER, MODIS, and MISR

NASA Technical Reports Server (NTRS)

King, Michael D.; Platnick, Steven; Wind, Galina; Arnold, George T.; Ackerman, Steven A.; Frey, Richard

2007-01-01

The MODIS Airborne Simulator (MAS) and MODIS/ASTER Airborne Simulator (MASTER) were used to obtain measurements of the bidirectional reflectance and brightness temperature of clouds at 50 discrete wavelengths between 0.47 and 14.3 (12.9 m for MASTER). These observations were obtained from the NASA ER-2 aircraft as part of the Tropical Composition, Clouds and Climate Coupling Experiment (TC4) conducted over Central America and surrounding Pacific and Atlantic Oceans between July 17 and August 8, 2007. Multispectral images in eight distinct bands were used to derive a confidence in clear sky (or alternatively the probability of cloud) over land and ocean ecosystems. Based on the results of individual tests run as part of this cloud mask, an algorithm was developed to estimate the phase of the clouds (liquid water, ice, or undetermined phase). Finally, the cloud optical thickness and effective radius were derived for both liquid water and ice clouds that were detected during each flight, using a nearly identical algorithm as that implemented operationally to process MODIS cloud data from the Aqua and Terra satellites (Collection 5). This analysis shows that the cloud mask developed for operational use on MODIS, and tested using MAS and MASTER date in TC4, is quite capable of distinguishing both liquid water and ice clouds during daytime conditions over both land and ocean. The cloud optical thickness and effective radius retrievals used three distinct bands of the MAS (or MASTER), and these results were compared with nearly simultaneous retrievals of MODIS on the Terra spacecraft. Finally, this MODIS-based algorithm was adapted to MISR data to infer the cloud optical thickness of liquid water clouds from MISR. Results of this analysis will be presented and discussed.

Road traffic sign detection and classification from mobile LiDAR point clouds

NASA Astrophysics Data System (ADS)

Weng, Shengxia; Li, Jonathan; Chen, Yiping; Wang, Cheng

2016-03-01

Traffic signs are important roadway assets that provide valuable information of the road for drivers to make safer and easier driving behaviors. Due to the development of mobile mapping systems that can efficiently acquire dense point clouds along the road, automated detection and recognition of road assets has been an important research issue. This paper deals with the detection and classification of traffic signs in outdoor environments using mobile light detection and ranging (Li- DAR) and inertial navigation technologies. The proposed method contains two main steps. It starts with an initial detection of traffic signs based on the intensity attributes of point clouds, as the traffic signs are always painted with highly reflective materials. Then, the classification of traffic signs is achieved based on the geometric shape and the pairwise 3D shape context. Some results and performance analyses are provided to show the effectiveness and limits of the proposed method. The experimental results demonstrate the feasibility and effectiveness of the proposed method in detecting and classifying traffic signs from mobile LiDAR point clouds.

An Efficient Optimization Method for Solving Unsupervised Data Classification Problems.

PubMed

Shabanzadeh, Parvaneh; Yusof, Rubiyah

2015-01-01

Unsupervised data classification (or clustering) analysis is one of the most useful tools and a descriptive task in data mining that seeks to classify homogeneous groups of objects based on similarity and is used in many medical disciplines and various applications. In general, there is no single algorithm that is suitable for all types of data, conditions, and applications. Each algorithm has its own advantages, limitations, and deficiencies. Hence, research for novel and effective approaches for unsupervised data classification is still active. In this paper a heuristic algorithm, Biogeography-Based Optimization (BBO) algorithm, was adapted for data clustering problems by modifying the main operators of BBO algorithm, which is inspired from the natural biogeography distribution of different species. Similar to other population-based algorithms, BBO algorithm starts with an initial population of candidate solutions to an optimization problem and an objective function that is calculated for them. To evaluate the performance of the proposed algorithm assessment was carried on six medical and real life datasets and was compared with eight well known and recent unsupervised data classification algorithms. Numerical results demonstrate that the proposed evolutionary optimization algorithm is efficient for unsupervised data classification.

3D Cloud Field Prediction using A-Train Data and Machine Learning Techniques

NASA Astrophysics Data System (ADS)

Johnson, C. L.

2017-12-01

Validation of cloud process parameterizations used in global climate models (GCMs) would greatly benefit from observed 3D cloud fields at the size comparable to that of a GCM grid cell. For the highest resolution simulations, surface grid cells are on the order of 100 km by 100 km. CloudSat/CALIPSO data provides 1 km width of detailed vertical cloud fraction profile (CFP) and liquid and ice water content (LWC/IWC). This work utilizes four machine learning algorithms to create nonlinear regressions of CFP, LWC, and IWC data using radiances, surface type and location of measurement as predictors and applies the regression equations to off-track locations generating 3D cloud fields for 100 km by 100 km domains. The CERES-CloudSat-CALIPSO-MODIS (C3M) merged data set for February 2007 is used. Support Vector Machines, Artificial Neural Networks, Gaussian Processes and Decision Trees are trained on 1000 km of continuous C3M data. Accuracy is computed using existing vertical profiles that are excluded from the training data and occur within 100 km of the training data. Accuracy of the four algorithms is compared. Average accuracy for one day of predicted data is 86% for the most successful algorithm. The methodology for training the algorithms, determining valid prediction regions and applying the equations off-track is discussed. Predicted 3D cloud fields are provided as inputs to the Ed4 NASA LaRC Fu-Liou radiative transfer code and resulting TOA radiances compared to observed CERES/MODIS radiances. Differences in computed radiances using predicted profiles and observed radiances are compared.

Gradient Evolution-based Support Vector Machine Algorithm for Classification

NASA Astrophysics Data System (ADS)

Zulvia, Ferani E.; Kuo, R. J.

2018-03-01

This paper proposes a classification algorithm based on a support vector machine (SVM) and gradient evolution (GE) algorithms. SVM algorithm has been widely used in classification. However, its result is significantly influenced by the parameters. Therefore, this paper aims to propose an improvement of SVM algorithm which can find the best SVMs’ parameters automatically. The proposed algorithm employs a GE algorithm to automatically determine the SVMs’ parameters. The GE algorithm takes a role as a global optimizer in finding the best parameter which will be used by SVM algorithm. The proposed GE-SVM algorithm is verified using some benchmark datasets and compared with other metaheuristic-based SVM algorithms. The experimental results show that the proposed GE-SVM algorithm obtains better results than other algorithms tested in this paper.

Comparison analysis for classification algorithm in data mining and the study of model use

NASA Astrophysics Data System (ADS)

Chen, Junde; Zhang, Defu

2018-04-01

As a key technique in data mining, classification algorithm was received extensive attention. Through an experiment of classification algorithm in UCI data set, we gave a comparison analysis method for the different algorithms and the statistical test was used here. Than that, an adaptive diagnosis model for preventive electricity stealing and leakage was given as a specific case in the paper.

myBlackBox: Blackbox Mobile Cloud Systems for Personalized Unusual Event Detection.

PubMed

Ahn, Junho; Han, Richard

2016-05-23

We demonstrate the feasibility of constructing a novel and practical real-world mobile cloud system, called myBlackBox, that efficiently fuses multimodal smartphone sensor data to identify and log unusual personal events in mobile users' daily lives. The system incorporates a hybrid architectural design that combines unsupervised classification of audio, accelerometer and location data with supervised joint fusion classification to achieve high accuracy, customization, convenience and scalability. We show the feasibility of myBlackBox by implementing and evaluating this end-to-end system that combines Android smartphones with cloud servers, deployed for 15 users over a one-month period.

myBlackBox: Blackbox Mobile Cloud Systems for Personalized Unusual Event Detection

PubMed Central

Ahn, Junho; Han, Richard

2016-01-01

We demonstrate the feasibility of constructing a novel and practical real-world mobile cloud system, called myBlackBox, that efficiently fuses multimodal smartphone sensor data to identify and log unusual personal events in mobile users’ daily lives. The system incorporates a hybrid architectural design that combines unsupervised classification of audio, accelerometer and location data with supervised joint fusion classification to achieve high accuracy, customization, convenience and scalability. We show the feasibility of myBlackBox by implementing and evaluating this end-to-end system that combines Android smartphones with cloud servers, deployed for 15 users over a one-month period. PMID:27223292

Performance of fusion algorithms for computer-aided detection and classification of mines in very shallow water obtained from testing in navy Fleet Battle Exercise-Hotel 2000

NASA Astrophysics Data System (ADS)

Ciany, Charles M.; Zurawski, William; Kerfoot, Ian

2001-10-01

The performance of Computer Aided Detection/Computer Aided Classification (CAD/CAC) Fusion algorithms on side-scan sonar images was evaluated using data taken at the Navy's's Fleet Battle Exercise-Hotel held in Panama City, Florida, in August 2000. A 2-of-3 binary fusion algorithm is shown to provide robust performance. The algorithm accepts the classification decisions and associated contact locations form three different CAD/CAC algorithms, clusters the contacts based on Euclidian distance, and then declares a valid target when a clustered contact is declared by at least 2 of the 3 individual algorithms. This simple binary fusion provided a 96 percent probability of correct classification at a false alarm rate of 0.14 false alarms per image per side. The performance represented a 3.8:1 reduction in false alarms over the best performing single CAD/CAC algorithm, with no loss in probability of correct classification.

Contributions to "k"-Means Clustering and Regression via Classification Algorithms

ERIC Educational Resources Information Center

Salman, Raied

2012-01-01

The dissertation deals with clustering algorithms and transforming regression problems into classification problems. The main contributions of the dissertation are twofold; first, to improve (speed up) the clustering algorithms and second, to develop a strict learning environment for solving regression problems as classification tasks by using…

Detecting Abnormal Machine Characteristics in Cloud Infrastructures

NASA Technical Reports Server (NTRS)

Bhaduri, Kanishka; Das, Kamalika; Matthews, Bryan L.

2011-01-01

In the cloud computing environment resources are accessed as services rather than as a product. Monitoring this system for performance is crucial because of typical pay-peruse packages bought by the users for their jobs. With the huge number of machines currently in the cloud system, it is often extremely difficult for system administrators to keep track of all machines using distributed monitoring programs such as Ganglia1 which lacks system health assessment and summarization capabilities. To overcome this problem, we propose a technique for automated anomaly detection using machine performance data in the cloud. Our algorithm is entirely distributed and runs locally on each computing machine on the cloud in order to rank the machines in order of their anomalous behavior for given jobs. There is no need to centralize any of the performance data for the analysis and at the end of the analysis, our algorithm generates error reports, thereby allowing the system administrators to take corrective actions. Experiments performed on real data sets collected for different jobs validate the fact that our algorithm has a low overhead for tracking anomalous machines in a cloud infrastructure.

A Comparative Study of YSO Classification Techniques using WISE Observations of the KR 120 Molecular Cloud

NASA Astrophysics Data System (ADS)

Kang, Sung-Ju; Kerton, C. R.

2014-01-01

KR 120 (Sh2-187) is a small Galactic HII region located at a distance of 1.4 kpc that shows evidence for triggered star formation in the surrounding molecular cloud. We present an analysis of the young stellar object (YSO) population of the molecular cloud as determined using a variety of classification techniques. YSO candidates are selected from the WISE all sky catalog and classified as Class I, Class II and Flat based on 1) spectral index, 2) color-color or color-magnitude plots, and 3) spectral energy distribution (SED) fits to radiative transfer models. We examine the discrepancies in YSO classification between the various techniques and explore how these discrepancies lead to uncertainty in such scientifically interesting quantities such as the ratio of Class I/Class II sources and the surface density of YSOs at various stages of evolution.

Ice crystals classification using airborne measurements in mixing phase

NASA Astrophysics Data System (ADS)

Sorin Vajaiac, Nicolae; Boscornea, Andreea

2017-04-01

This paper presents a case study of ice crystals classification from airborne measurements in mixed-phase clouds. Ice crystal shadow is recorded with CIP (Cloud Imaging Probe) component of CAPS (Cloud, Aerosol, and Precipitation Spectrometer) system. The analyzed flight was performed in the south-western part of Romania (between Pietrosani, Ramnicu Valcea, Craiova and Targu Jiu), with a Beechcraft C90 GTX which was specially equipped with a CAPS system. The temperature, during the fly, reached the lowest value at -35 °C. These low temperatures allow the formation of ice crystals and influence their form. For the here presented ice crystals classification a special software, OASIS (Optical Array Shadow Imaging Software), developed by DMT (Droplet Measurement Technologies), was used. The obtained results, as expected are influenced by the atmospheric and microphysical parameters. The particles recorded where classified in four groups: edge, irregular, round and small.

SHOCKFIND - an algorithm to identify magnetohydrodynamic shock waves in turbulent clouds

NASA Astrophysics Data System (ADS)

Lehmann, Andrew; Federrath, Christoph; Wardle, Mark

2016-11-01

The formation of stars occurs in the dense molecular cloud phase of the interstellar medium. Observations and numerical simulations of molecular clouds have shown that supersonic magnetized turbulence plays a key role for the formation of stars. Simulations have also shown that a large fraction of the turbulent energy dissipates in shock waves. The three families of MHD shocks - fast, intermediate and slow - distinctly compress and heat up the molecular gas, and so provide an important probe of the physical conditions within a turbulent cloud. Here, we introduce the publicly available algorithm, SHOCKFIND, to extract and characterize the mixture of shock families in MHD turbulence. The algorithm is applied to a three-dimensional simulation of a magnetized turbulent molecular cloud, and we find that both fast and slow MHD shocks are present in the simulation. We give the first prediction of the mixture of turbulence-driven MHD shock families in this molecular cloud, and present their distinct distributions of sonic and Alfvénic Mach numbers. Using subgrid one-dimensional models of MHD shocks we estimate that ˜0.03 per cent of the volume of a typical molecular cloud in the Milky Way will be shock heated above 50 K, at any time during the lifetime of the cloud. We discuss the impact of this shock heating on the dynamical evolution of molecular clouds.

Added Value of Far-Infrared Radiometry for Ice Cloud Remote Sensing

NASA Astrophysics Data System (ADS)

Libois, Q.; Blanchet, J. P.; Ivanescu, L.; S Pelletier, L.; Laurence, C.

2017-12-01

Several cloud retrieval algorithms based on satellite observations in the infrared have been developed in the last decades. However, most of these observations only cover the midinfrared (MIR, λ < 15 μm) part of the spectrum, and none are available in the far-infrared (FIR, λ ≥ 15 μm). Recent developments in FIR sensors technology, though, now make it possible to consider spaceborne remote sensing in the FIR. Here we show that adding a few FIR channels with realistic radiometric performances to existing spaceborne narrowband radiometers would significantly improve their ability to retrieve ice cloud radiative properties. For clouds encountered in the polar regions and the upper troposphere, where the atmosphere above clouds is sufficiently transparent in the FIR, using FIR channels would reduce by more than 50% the uncertainties on retrieved values of optical thickness, effective particle diameter, and cloud top altitude. This would somehow extend the range of applicability of current infrared retrieval methods to the polar regions and to clouds with large optical thickness, where MIR algorithms perform poorly. The high performance of solar reflection-based algorithms would thus be reached in nighttime conditions. Using FIR observations is a promising venue for studying ice cloud microphysics and precipitation processes, which is highly relevant for cirrus clouds and convective towers, and for investigating the water cycle in the driest regions of the atmosphere.

A Modified Decision Tree Algorithm Based on Genetic Algorithm for Mobile User Classification Problem

PubMed Central

Liu, Dong-sheng; Fan, Shu-jiang

2014-01-01

In order to offer mobile customers better service, we should classify the mobile user firstly. Aimed at the limitations of previous classification methods, this paper puts forward a modified decision tree algorithm for mobile user classification, which introduced genetic algorithm to optimize the results of the decision tree algorithm. We also take the context information as a classification attributes for the mobile user and we classify the context into public context and private context classes. Then we analyze the processes and operators of the algorithm. At last, we make an experiment on the mobile user with the algorithm, we can classify the mobile user into Basic service user, E-service user, Plus service user, and Total service user classes and we can also get some rules about the mobile user. Compared to C4.5 decision tree algorithm and SVM algorithm, the algorithm we proposed in this paper has higher accuracy and more simplicity. PMID:24688389

Efficient terrestrial laser scan segmentation exploiting data structure

NASA Astrophysics Data System (ADS)

Mahmoudabadi, Hamid; Olsen, Michael J.; Todorovic, Sinisa

2016-09-01

New technologies such as lidar enable the rapid collection of massive datasets to model a 3D scene as a point cloud. However, while hardware technology continues to advance, processing 3D point clouds into informative models remains complex and time consuming. A common approach to increase processing efficiently is to segment the point cloud into smaller sections. This paper proposes a novel approach for point cloud segmentation using computer vision algorithms to analyze panoramic representations of individual laser scans. These panoramas can be quickly created using an inherent neighborhood structure that is established during the scanning process, which scans at fixed angular increments in a cylindrical or spherical coordinate system. In the proposed approach, a selected image segmentation algorithm is applied on several input layers exploiting this angular structure including laser intensity, range, normal vectors, and color information. These segments are then mapped back to the 3D point cloud so that modeling can be completed more efficiently. This approach does not depend on pre-defined mathematical models and consequently setting parameters for them. Unlike common geometrical point cloud segmentation methods, the proposed method employs the colorimetric and intensity data as another source of information. The proposed algorithm is demonstrated on several datasets encompassing variety of scenes and objects. Results show a very high perceptual (visual) level of segmentation and thereby the feasibility of the proposed algorithm. The proposed method is also more efficient compared to Random Sample Consensus (RANSAC), which is a common approach for point cloud segmentation.

Cloud-based Predictive Modeling System and its Application to Asthma Readmission Prediction

PubMed Central

Chen, Robert; Su, Hang; Khalilia, Mohammed; Lin, Sizhe; Peng, Yue; Davis, Tod; Hirsh, Daniel A; Searles, Elizabeth; Tejedor-Sojo, Javier; Thompson, Michael; Sun, Jimeng

2015-01-01

The predictive modeling process is time consuming and requires clinical researchers to handle complex electronic health record (EHR) data in restricted computational environments. To address this problem, we implemented a cloud-based predictive modeling system via a hybrid setup combining a secure private server with the Amazon Web Services (AWS) Elastic MapReduce platform. EHR data is preprocessed on a private server and the resulting de-identified event sequences are hosted on AWS. Based on user-specified modeling configurations, an on-demand web service launches a cluster of Elastic Compute 2 (EC2) instances on AWS to perform feature selection and classification algorithms in a distributed fashion. Afterwards, the secure private server aggregates results and displays them via interactive visualization. We tested the system on a pediatric asthma readmission task on a de-identified EHR dataset of 2,967 patients. We conduct a larger scale experiment on the CMS Linkable 2008–2010 Medicare Data Entrepreneurs’ Synthetic Public Use File dataset of 2 million patients, which achieves over 25-fold speedup compared to sequential execution. PMID:26958172

Improved retrieval of cloud base heights from ceilometer using a non-standard instrument method

NASA Astrophysics Data System (ADS)

Wang, Yang; Zhao, Chuanfeng; Dong, Zipeng; Li, Zhanqing; Hu, Shuzhen; Chen, Tianmeng; Tao, Fa; Wang, Yuzhao

2018-04-01

Cloud-base height (CBH) is a basic cloud parameter but has not been measured accurately, especially under polluted conditions due to the interference of aerosol. Taking advantage of a comprehensive field experiment in northern China in which a variety of advanced cloud probing instruments were operated, different methods of detecting CBH are assessed. The Micro-Pulse Lidar (MPL) and the Vaisala ceilometer (CL51) provided two types of backscattered profiles. The latter has been employed widely as a standard means of measuring CBH using the manufacturer's operational algorithm to generate standard CBH products (CL51 MAN) whose quality is rigorously assessed here, in comparison with a research algorithm that we developed named value distribution equalization (VDE) algorithm. It was applied to both the profiles of lidar backscattering data from the two instruments. The VDE algorithm is found to produce more accurate estimates of CBH for both instruments and can cope with heavy aerosol loading conditions well. By contrast, CL51 MAN overestimates CBH by 400 m and misses many low level clouds under such conditions. These findings are important given that CL51 has been adopted operationally by many meteorological stations in China.

Heading Toward Launch with the Integrated Multi-Satellite Retrievals for GPM (IMERG)

NASA Technical Reports Server (NTRS)

Huffman, George J.; Bolvin, David T.; Nelkin, Eric J.; Adler, Robert F.

2012-01-01

The Day-l algorithm for computing combined precipitation estimates in GPM is the Integrated Multi-satellitE Retrievals for GPM (IMERG). We plan for the period of record to encompass both the TRMM and GPM eras, and the coverage to extend to fully global as experience is gained in the difficult high-latitude environment. IMERG is being developed as a unified U.S. algorithm that takes advantage of strengths in the three groups that are contributing expertise: 1) the TRMM Multi-satellite Precipitation Analysis (TMPA), which addresses inter-satellite calibration of precipitation estimates and monthly scale combination of satellite and gauge analyses; 2) the CPC Morphing algorithm with Kalman Filtering (KF-CMORPH), which provides quality-weighted time interpolation of precipitation patterns following cloud motion; and 3) the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks using a Cloud Classification System (PERSIANN-CCS), which provides a neural-network-based scheme for generating microwave-calibrated precipitation estimates from geosynchronous infrared brightness temperatures. In this talk we summarize the major building blocks and important design issues driven by user needs and practical data issues. One concept being pioneered by the IMERG team is that the code system should produce estimates for the same time period but at different latencies to support the requirements of different groups of users. Another user requirement is that all these runs must be reprocessed as new IMERG versions are introduced. IMERG's status at meeting time will be summarized, and the processing scenario in the transition from TRMM to GPM will be laid out. Initially, IMERG will be run with TRMM-based calibration, and then a conversion to a GPM-based calibration will be employed after the GPM sensor products are validated. A complete reprocessing will be computed, which will complete the transition from TMPA.

Longitudinal Control for Mengshi Autonomous Vehicle via Cloud Model

NASA Astrophysics Data System (ADS)

Gao, H. B.; Zhang, X. Y.; Li, D. Y.; Liu, Y. C.

2018-03-01

Dynamic robustness and stability control is a requirement for self-driving of autonomous vehicle. Longitudinal control method of autonomous is a key technique which has drawn the attention of industry and academe. In this paper, we present a longitudinal control algorithm based on cloud model for Mengshi autonomous vehicle to ensure the dynamic stability and tracking performance of Mengshi autonomous vehicle. An experiments is applied to test the implementation of the longitudinal control algorithm. Empirical results show that if the longitudinal control algorithm based Gauss cloud model are applied to calculate the acceleration, and the vehicles drive at different speeds, a stable longitudinal control effect is achieved.

LIVAS: a 3-D multi-wavelength aerosol/cloud database based on CALIPSO and EARLINET

NASA Astrophysics Data System (ADS)

Amiridis, V.; Marinou, E.; Tsekeri, A.; Wandinger, U.; Schwarz, A.; Giannakaki, E.; Mamouri, R.; Kokkalis, P.; Binietoglou, I.; Solomos, S.; Herekakis, T.; Kazadzis, S.; Gerasopoulos, E.; Proestakis, E.; Kottas, M.; Balis, D.; Papayannis, A.; Kontoes, C.; Kourtidis, K.; Papagiannopoulos, N.; Mona, L.; Pappalardo, G.; Le Rille, O.; Ansmann, A.

2015-07-01

We present LIVAS (LIdar climatology of Vertical Aerosol Structure for space-based lidar simulation studies), a 3-D multi-wavelength global aerosol and cloud optical database, optimized to be used for future space-based lidar end-to-end simulations of realistic atmospheric scenarios as well as retrieval algorithm testing activities. The LIVAS database provides averaged profiles of aerosol optical properties for the potential spaceborne laser operating wavelengths of 355, 532, 1064, 1570 and 2050 nm and of cloud optical properties at the wavelength of 532 nm. The global database is based on CALIPSO observations at 532 and 1064 nm and on aerosol-type-dependent backscatter- and extinction-related Ångström exponents, derived from EARLINET (European Aerosol Research Lidar Network) ground-based measurements for the UV and scattering calculations for the IR wavelengths, using a combination of input data from AERONET, suitable aerosol models and recent literature. The required spectral conversions are calculated for each of the CALIPSO aerosol types and are applied to CALIPSO backscatter and extinction data corresponding to the aerosol type retrieved by the CALIPSO aerosol classification scheme. A cloud optical database based on CALIPSO measurements at 532 nm is also provided, neglecting wavelength conversion due to approximately neutral scattering behavior of clouds along the spectral range of LIVAS. Averages of particle linear depolarization ratio profiles at 532 nm are provided as well. Finally, vertical distributions for a set of selected scenes of specific atmospheric phenomena (e.g., dust outbreaks, volcanic eruptions, wild fires, polar stratospheric clouds) are analyzed and spectrally converted so as to be used as case studies for spaceborne lidar performance assessments. The final global data set includes 4-year (1 January 2008-31 December 2011) time-averaged CALIPSO (Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations) data on a uniform grid of 1° × 1° with the original high vertical resolution of CALIPSO in order to ensure realistic simulations of the atmospheric variability in lidar end-to-end simulations.

The MODIS Cloud Optical and Microphysical Products: Collection 6 Up-dates and Examples From Terra and Aqua

NASA Technical Reports Server (NTRS)

Platnick, Steven; Meyer, Kerry G.; King, Michael D.; Wind, Galina; Amarasinghe, Nandana; Marchant, Benjamin G.; Arnold, G. Thomas; Zhang, Zhibo; Hubanks, Paul A.; Holz, Robert E.;

The MODIS cloud optical and microphysical products: Collection 6 updates and examples from Terra and Aqua.

PubMed

Platnick, Steven; Meyer, Kerry G; King, Michael D; Wind, Galina; Amarasinghe, Nandana; Marchant, Benjamin; Arnold, G Thomas; Zhang, Zhibo; Hubanks, Paul A; Holz, Robert E; Yang, Ping; Ridgway, William L; Riedi, Jérôme

2017-01-01

The MODIS Level-2 cloud product (Earth Science Data Set names MOD06 and MYD06 for Terra and Aqua MODIS, respectively) provides pixel-level retrievals of cloud-top properties (day and night pressure, temperature, and height) and cloud optical properties (optical thickness, effective particle radius, and water path for both liquid water and ice cloud thermodynamic phases-daytime only). Collection 6 (C6) reprocessing of the product was completed in May 2014 and March 2015 for MODIS Aqua and Terra, respectively. Here we provide an overview of major C6 optical property algorithm changes relative to the previous Collection 5 (C5) product. Notable C6 optical and microphysical algorithm changes include: (i) new ice cloud optical property models and a more extensive cloud radiative transfer code lookup table (LUT) approach, (ii) improvement in the skill of the shortwave-derived cloud thermodynamic phase, (iii) separate cloud effective radius retrieval datasets for each spectral combination used in previous collections, (iv) separate retrievals for partly cloudy pixels and those associated with cloud edges, (v) failure metrics that provide diagnostic information for pixels having observations that fall outside the LUT solution space, and (vi) enhanced pixel-level retrieval uncertainty calculations. The C6 algorithm changes collectively can result in significant changes relative to C5, though the magnitude depends on the dataset and the pixel's retrieval location in the cloud parameter space. Example Level-2 granule and Level-3 gridded dataset differences between the two collections are shown. While the emphasis is on the suite of cloud optical property datasets, other MODIS cloud datasets are discussed when relevant.

The MODIS cloud optical and microphysical products: Collection 6 updates and examples from Terra and Aqua

PubMed Central

Platnick, Steven; Meyer, Kerry G.; King, Michael D.; Wind, Galina; Amarasinghe, Nandana; Marchant, Benjamin; Arnold, G. Thomas; Zhang, Zhibo; Hubanks, Paul A.; Holz, Robert E.; Yang, Ping; Ridgway, William L.; Riedi, Jérôme

2018-01-01

The MODIS Level-2 cloud product (Earth Science Data Set names MOD06 and MYD06 for Terra and Aqua MODIS, respectively) provides pixel-level retrievals of cloud-top properties (day and night pressure, temperature, and height) and cloud optical properties (optical thickness, effective particle radius, and water path for both liquid water and ice cloud thermodynamic phases–daytime only). Collection 6 (C6) reprocessing of the product was completed in May 2014 and March 2015 for MODIS Aqua and Terra, respectively. Here we provide an overview of major C6 optical property algorithm changes relative to the previous Collection 5 (C5) product. Notable C6 optical and microphysical algorithm changes include: (i) new ice cloud optical property models and a more extensive cloud radiative transfer code lookup table (LUT) approach, (ii) improvement in the skill of the shortwave-derived cloud thermodynamic phase, (iii) separate cloud effective radius retrieval datasets for each spectral combination used in previous collections, (iv) separate retrievals for partly cloudy pixels and those associated with cloud edges, (v) failure metrics that provide diagnostic information for pixels having observations that fall outside the LUT solution space, and (vi) enhanced pixel-level retrieval uncertainty calculations. The C6 algorithm changes collectively can result in significant changes relative to C5, though the magnitude depends on the dataset and the pixel’s retrieval location in the cloud parameter space. Example Level-2 granule and Level-3 gridded dataset differences between the two collections are shown. While the emphasis is on the suite of cloud optical property datasets, other MODIS cloud datasets are discussed when relevant. PMID:29657349

Sensitivity of Marine Warm Cloud Retrieval Statistics to Algorithm Choices: Examples from MODIS Collection 6

NASA Technical Reports Server (NTRS)

Platnick, Steven; Wind, Galina; Zhang, Zhibo; Ackerman, Steven A.; Maddux, Brent

2012-01-01

The optical and microphysical structure of warm boundary layer marine clouds is of fundamental importance for understanding a variety of cloud radiation and precipitation processes. With the advent of MODIS (Moderate Resolution Imaging Spectroradiometer) on the NASA EOS Terra and Aqua platforms, simultaneous global/daily 1km retrievals of cloud optical thickness and effective particle size are provided, as well as the derived water path. In addition, the cloud product (MOD06/MYD06 for MODIS Terra and Aqua, respectively) provides separate effective radii results using the l.6, 2.1, and 3.7 m spectral channels. Cloud retrieval statistics are highly sensitive to how a pixel identified as being "notclear" by a cloud mask (e.g., the MOD35/MYD35 product) is determined to be useful for an optical retrieval based on a 1-D cloud model. The Collection 5 MODIS retrieval algorithm removed pixels associated with cloud'edges as well as ocean pixels with partly cloudy elements in the 250m MODIS cloud mask - part of the so-called Clear Sky Restoral (CSR) algorithm. Collection 6 attempts retrievals for those two pixel populations, but allows a user to isolate or filter out the populations via CSR pixel-level Quality Assessment (QA) assignments. In this paper, using the preliminary Collection 6 MOD06 product, we present global and regional statistical results of marine warm cloud retrieval sensitivities to the cloud edge and 250m partly cloudy pixel populations. As expected, retrievals for these pixels are generally consistent with a breakdown of the ID cloud model. While optical thickness for these suspect pixel populations may have some utility for radiative studies, the retrievals should be used with extreme caution for process and microphysical studies.

Big data mining analysis method based on cloud computing

NASA Astrophysics Data System (ADS)

Cai, Qing Qiu; Cui, Hong Gang; Tang, Hao

2017-08-01

Information explosion era, large data super-large, discrete and non-(semi) structured features have gone far beyond the traditional data management can carry the scope of the way. With the arrival of the cloud computing era, cloud computing provides a new technical way to analyze the massive data mining, which can effectively solve the problem that the traditional data mining method cannot adapt to massive data mining. This paper introduces the meaning and characteristics of cloud computing, analyzes the advantages of using cloud computing technology to realize data mining, designs the mining algorithm of association rules based on MapReduce parallel processing architecture, and carries out the experimental verification. The algorithm of parallel association rule mining based on cloud computing platform can greatly improve the execution speed of data mining.

Classification of Dual-Wavelength Airborne Laser Scanning Point Cloud Based on the Radiometric Properties of the Objects

NASA Astrophysics Data System (ADS)

Pilarska, M.

2018-05-01

Airborne laser scanning (ALS) is a well-known and willingly used technology. One of the advantages of this technology is primarily its fast and accurate data registration. In recent years ALS is continuously developed. One of the latest achievements is multispectral ALS, which consists in obtaining simultaneously the data in more than one laser wavelength. In this article the results of the dual-wavelength ALS data classification are presented. The data were acquired with RIEGL VQ-1560i sensor, which is equipped with two laser scanners operating in different wavelengths: 532 nm and 1064 nm. Two classification approaches are presented in the article: classification, which is based on geometric relationships between points and classification, which mostly relies on the radiometric properties of registered objects. The overall accuracy of the geometric classification was 86 %, whereas for the radiometric classification it was 81 %. As a result, it can be assumed that the radiometric features which are provided by the multispectral ALS have potential to be successfully used in ALS point cloud classification.

Improved classification accuracy by feature extraction using genetic algorithms

NASA Astrophysics Data System (ADS)

Patriarche, Julia; Manduca, Armando; Erickson, Bradley J.

2003-05-01

A feature extraction algorithm has been developed for the purposes of improving classification accuracy. The algorithm uses a genetic algorithm / hill-climber hybrid to generate a set of linearly recombined features, which may be of reduced dimensionality compared with the original set. The genetic algorithm performs the global exploration, and a hill climber explores local neighborhoods. Hybridizing the genetic algorithm with a hill climber improves both the rate of convergence, and the final overall cost function value; it also reduces the sensitivity of the genetic algorithm to parameter selection. The genetic algorithm includes the operators: crossover, mutation, and deletion / reactivation - the last of these effects dimensionality reduction. The feature extractor is supervised, and is capable of deriving a separate feature space for each tissue (which are reintegrated during classification). A non-anatomical digital phantom was developed as a gold standard for testing purposes. In tests with the phantom, and with images of multiple sclerosis patients, classification with feature extractor derived features yielded lower error rates than using standard pulse sequences, and with features derived using principal components analysis. Using the multiple sclerosis patient data, the algorithm resulted in a mean 31% reduction in classification error of pure tissues.

Preliminary verification for application of a support vector machine-based cloud detection method to GOSAT-2 CAI-2

NASA Astrophysics Data System (ADS)

Oishi, Yu; Ishida, Haruma; Nakajima, Takashi Y.; Nakamura, Ryosuke; Matsunaga, Tsuneo

2018-05-01

The Greenhouse Gases Observing Satellite (GOSAT) was launched in 2009 to measure global atmospheric CO2 and CH4 concentrations. GOSAT is equipped with two sensors: the Thermal And Near infrared Sensor for carbon Observations (TANSO)-Fourier transform spectrometer (FTS) and TANSO-Cloud and Aerosol Imager (CAI). The presence of clouds in the instantaneous field of view of the FTS leads to incorrect estimates of the concentrations. Thus, the FTS data suspected to have cloud contamination must be identified by a CAI cloud discrimination algorithm and rejected. Conversely, overestimating clouds reduces the amount of FTS data that can be used to estimate greenhouse gas concentrations. This is a serious problem in tropical rainforest regions, such as the Amazon, where the amount of useable FTS data is small because of cloud cover. Preparations are continuing for the launch of the GOSAT-2 in fiscal year 2018. To improve the accuracy of the estimates of greenhouse gases concentrations, we need to refine the existing CAI cloud discrimination algorithm: Cloud and Aerosol Unbiased Decision Intellectual Algorithm (CLAUDIA1). A new cloud discrimination algorithm using a support vector machine (CLAUDIA3) was developed and presented in another paper. Although the use of visual inspection of clouds as a standard for judging is not practical for screening a full satellite data set, it has the advantage of allowing for locally optimized thresholds, while CLAUDIA1 and -3 use common global thresholds. Thus, the accuracy of visual inspection is better than that of these algorithms in most regions, with the exception of snow- and ice-covered surfaces, where there is not enough spectral contrast to identify cloud. In other words, visual inspection results can be used as truth data for accuracy evaluation of CLAUDIA1 and -3. For this reason visual inspection can be used for the truth metric for the cloud discrimination verification exercise. In this study, we compared CLAUDIA1-CAI and CLAUDIA3-CAI for various land cover types, and evaluated the accuracy of CLAUDIA3-CAI by comparing both CLAUDIA1-CAI and CLAUDIA3-CAI with visual inspection (400 × 400 pixels) of the same CAI images in tropical rainforests. Comparative results between CLAUDIA1-CAI and CLAUDIA3-CAI for various land cover types indicated that CLAUDIA3-CAI had a tendency to identify bright surface and optically thin clouds. However, CLAUDIA3-CAI had a tendency to misjudge the edges of clouds compared with CLAUDIA1-CAI. The accuracy of CLAUDIA3-CAI was approximately 89.5 % in tropical rainforests, which is greater than that of CLAUDIA1-CAI (85.9 %) for the test cases presented here.

[A cloud detection algorithm for MODIS images combining Kmeans clustering and multi-spectral threshold method].

PubMed

Wang, Wei; Song, Wei-Guo; Liu, Shi-Xing; Zhang, Yong-Ming; Zheng, Hong-Yang; Tian, Wei

2011-04-01

An improved method for detecting cloud combining Kmeans clustering and the multi-spectral threshold approach is described. On the basis of landmark spectrum analysis, MODIS data is categorized into two major types initially by Kmeans method. The first class includes clouds, smoke and snow, and the second class includes vegetation, water and land. Then a multi-spectral threshold detection is applied to eliminate interference such as smoke and snow for the first class. The method is tested with MODIS data at different time under different underlying surface conditions. By visual method to test the performance of the algorithm, it was found that the algorithm can effectively detect smaller area of cloud pixels and exclude the interference of underlying surface, which provides a good foundation for the next fire detection approach.

Accuracy of Geophysical Parameters Derived from AIRS/AMSU as a Function of Fractional Cloud Cover

NASA Technical Reports Server (NTRS)

Susskind, Joel; Barnet, Chris; Blaisdell, John; Iredell, Lena; Keita, Fricky; Kouvaris, Lou; Molnar, Gyula; Chahine, Moustafa

2005-01-01

AIRS was launched on EOS Aqua on May 4,2002, together with AMSU A and HSB, to form a next generation polar orbiting infrared and microwave atmospheric sounding system. The primary products of AIRS/AMSU are twice daily global fields of atmospheric temperature-humidity profiles, ozone profiles, sea/land surface skin temperature, and cloud related parameters including OLR. The sounding goals of AIRS are to produce 1 km tropospheric layer mean temperatures with an rms error of 1K, and layer precipitable water with an rms error of 20%, in cases with up to 80% effective cloud cover. The basic theory used to analyze AIRS/AMSU/HSB data in the presence of clouds, called the at-launch algorithm, was described previously. Pre-launch simulation studies using this algorithm indicated that these results should be achievable. Some modifications have been made to the at-launch retrieval algorithm as described in this paper. Sample fields of parameters retrieved from AIRS/AMSU/HSB data are presented and validated as a function of retrieved fractional cloud cover. As in simulation, the degradation of retrieval accuracy with increasing cloud cover is small. HSB failed in February 2005, and consequently HSB channel radiances are not used in the results shown in this paper. The AIRS/AMSU retrieval algorithm described in this paper, called Version 4, become operational at the Goddard DAAC in April 2005 and is being used to analyze near-real time AIRS/AMSU data. Historical AIRS/AMSU data, going backwards from March 2005 through September 2002, is also being analyzed by the DAAC using the Version 4 algorithm.

Cloud vertical profiles derived from CALIPSO and CloudSat and a comparison with MODIS derived clouds

NASA Astrophysics Data System (ADS)

Kato, S.; Sun-Mack, S.; Miller, W. F.; Rose, F. G.; Minnis, P.; Wielicki, B. A.; Winker, D. M.; Stephens, G. L.; Charlock, T. P.; Collins, W. D.; Loeb, N. G.; Stackhouse, P. W.; Xu, K.

2008-05-01

CALIPSO and CloudSat from the a-train provide detailed information of vertical distribution of clouds and aerosols. The vertical distribution of cloud occurrence is derived from one month of CALIPSO and CloudSat data as a part of the effort of merging CALIPSO, CloudSat and MODIS with CERES data. This newly derived cloud profile is compared with the distribution of cloud top height derived from MODIS on Aqua from cloud algorithms used in the CERES project. The cloud base from MODIS is also estimated using an empirical formula based on the cloud top height and optical thickness, which is used in CERES processes. While MODIS detects mid and low level clouds over the Arctic in April fairly well when they are the topmost cloud layer, it underestimates high- level clouds. In addition, because the CERES-MODIS cloud algorithm is not able to detect multi-layer clouds and the empirical formula significantly underestimates the depth of high clouds, the occurrence of mid and low-level clouds is underestimated. This comparison does not consider sensitivity difference to thin clouds but we will impose an optical thickness threshold to CALIPSO derived clouds for a further comparison. The effect of such differences in the cloud profile to flux computations will also be discussed. In addition, the effect of cloud cover to the top-of-atmosphere flux over the Arctic using CERES SSF and FLASHFLUX products will be discussed.

The Q Continuum: Encounter with the Cloud Mask

NASA Astrophysics Data System (ADS)

Ackerman, S. A.; Frey, R.; Holz, R.; Philips, C.; Dutcher, S.

2017-12-01

We are developing a common cloud mask for MODIS and VIIRS observations, referred to as the MODIS VIIRS Continuity Mask (MVCM). Our focus is on extending the MODIS-heritage cloud detection approach in order to generate appropriate climate data records for clouds and climate studies. The MVCM is based on heritage from the MODIS cloud mask (MOD35 and MYD35) and employs a series of tests on MODIS reflectances and brightness temperatures. Cloud detection is based on contrasts (i.e., cloud versus background surface) at pixel resolution. The MVCM follows the same approach. These cloud masks use multiple cloud detection tests to indicate the confidence level that the observation is of a clear-sky scene. The outcome of a test ranges from 0 (cloudy) to 1 (clear-sky scene). Because of overlap in the sensitivities of the various spectral tests to the type of cloud, each test is considered in one of several groups. The final cloud mask is determined from the product of the minimum confidence of each group and is referred to as the Q value as defined in Ackerman et al (1998). In MOD35 and MYD35 processing, the Q value is not output, rather predetermined Q values determine the result: If Q ≥ .99 the scene is clear; .95 ≤ Q < .99 the pixel is probably a clear scene, .66 ≤ Q < .95 is probably cloudy and Q < .66 is cloudy. Thus representing Q discretely and not as a continuum. For the MVCM, the numerical value of the Q is output along with the classification of clear, probably clear, probably cloudy, and cloudy. Through comparisons with collocated CALIOP and MODIS observations, we will assess the categorization of the Q values as a function of scene type ). While validation studies have indicated the utility and statistical correctness of the cloud mask approach, the algorithm does not possess immeasurable power and perfection. This comparison will assess the time and space dependence of Q and assure that the laws of physics are followed, at least according to normal human notions. Using CALIOP as representing truth, a receiver operating characteristic curve (ROC) will be analyzed to determine the optimum Q for various scenes and seasons, thus providing a continuum of discriminating thresholds.

Development of Time-Series Human Settlement Mapping System Using Historical Landsat Archive

NASA Astrophysics Data System (ADS)

Miyazaki, H.; Nagai, M.; Shibasaki, R.

2016-06-01

Methodology of automated human settlement mapping is highly needed for utilization of historical satellite data archives for urgent issues of urban growth in global scale, such as disaster risk management, public health, food security, and urban management. As development of global data with spatial resolution of 10-100 m was achieved by some initiatives using ASTER, Landsat, and TerraSAR-X, next goal has targeted to development of time-series data which can contribute to studies urban development with background context of socioeconomy, disaster risk management, public health, transport and other development issues. We developed an automated algorithm to detect human settlement by classification of built-up and non-built-up in time-series Landsat images. A machine learning algorithm, Local and Global Consistency (LLGC), was applied with improvements for remote sensing data. The algorithm enables to use MCD12Q1, a MODIS-based global land cover map with 500-m resolution, as training data so that any manual process is not required for preparation of training data. In addition, we designed the method to composite multiple results of LLGC into a single output to reduce uncertainty. The LLGC results has a confidence value ranging 0.0 to 1.0 representing probability of built-up and non-built-up. The median value of the confidence for a certain period around a target time was expected to be a robust output of confidence to identify built-up or non-built-up areas against uncertainties in satellite data quality, such as cloud and haze contamination. Four scenes of Landsat data for each target years, 1990, 2000, 2005, and 2010, were chosen among the Landsat archive data with cloud contamination less than 20%.We developed a system with the algorithms on the Data Integration and Analysis System (DIAS) in the University of Tokyo and processed 5200 scenes of Landsat data for cities with more than one million people worldwide.

Contextual classification of multispectral image data: Approximate algorithm

NASA Technical Reports Server (NTRS)

Tilton, J. C. (Principal Investigator)

1980-01-01

An approximation to a classification algorithm incorporating spatial context information in a general, statistical manner is presented which is computationally less intensive. Classifications that are nearly as accurate are produced.

Towards a robust framework for catchment classification

NASA Astrophysics Data System (ADS)

Deshmukh, A.; Samal, A.; Singh, R.

2017-12-01

Classification of catchments based on various measures of similarity has emerged as an important technique to understand regional scale hydrologic behavior. Classification of catchment characteristics and/or streamflow response has been used reveal which characteristics are more likely to explain the observed variability of hydrologic response. However, numerous algorithms for supervised or unsupervised classification are available, making it hard to identify the algorithm most suitable for the dataset at hand. Consequently, existing catchment classification studies vary significantly in the classification algorithms employed with no previous attempt at understanding the degree of uncertainty in classification due to this algorithmic choice. This hinders the generalizability of interpretations related to hydrologic behavior. Our goal is to develop a protocol that can be followed while classifying hydrologic datasets. We focus on a classification framework for unsupervised classification and provide a step-by-step classification procedure. The steps include testing the clusterabiltiy of original dataset prior to classification, feature selection, validation of clustered data, and quantification of similarity of two clusterings. We test several commonly available methods within this framework to understand the level of similarity of classification results across algorithms. We apply the proposed framework on recently developed datasets for India to analyze to what extent catchment properties can explain observed catchment response. Our testing dataset includes watershed characteristics for over 200 watersheds which comprise of both natural (physio-climatic) characteristics and socio-economic characteristics. This framework allows us to understand the controls on observed hydrologic variability across India.

Comparison of four machine learning algorithms for their applicability in satellite-based optical rainfall retrievals

NASA Astrophysics Data System (ADS)

Meyer, Hanna; Kühnlein, Meike; Appelhans, Tim; Nauss, Thomas

2016-03-01

Machine learning (ML) algorithms have successfully been demonstrated to be valuable tools in satellite-based rainfall retrievals which show the practicability of using ML algorithms when faced with high dimensional and complex data. Moreover, recent developments in parallel computing with ML present new possibilities for training and prediction speed and therefore make their usage in real-time systems feasible. This study compares four ML algorithms - random forests (RF), neural networks (NNET), averaged neural networks (AVNNET) and support vector machines (SVM) - for rainfall area detection and rainfall rate assignment using MSG SEVIRI data over Germany. Satellite-based proxies for cloud top height, cloud top temperature, cloud phase and cloud water path serve as predictor variables. The results indicate an overestimation of rainfall area delineation regardless of the ML algorithm (averaged bias = 1.8) but a high probability of detection ranging from 81% (SVM) to 85% (NNET). On a 24-hour basis, the performance of the rainfall rate assignment yielded R2 values between 0.39 (SVM) and 0.44 (AVNNET). Though the differences in the algorithms' performance were rather small, NNET and AVNNET were identified as the most suitable algorithms. On average, they demonstrated the best performance in rainfall area delineation as well as in rainfall rate assignment. NNET's computational speed is an additional advantage in work with large datasets such as in remote sensing based rainfall retrievals. However, since no single algorithm performed considerably better than the others we conclude that further research in providing suitable predictors for rainfall is of greater necessity than an optimization through the choice of the ML algorithm.

Daytime sea fog retrieval based on GOCI data: a case study over the Yellow Sea.

PubMed

Yuan, Yibo; Qiu, Zhongfeng; Sun, Deyong; Wang, Shengqiang; Yue, Xiaoyuan

2016-01-25

In this paper, a new daytime sea fog detection algorithm has been developed by using Geostationary Ocean Color Imager (GOCI) data. Based on spectral analysis, differences in spectral characteristics were found over different underlying surfaces, which include land, sea, middle/high level clouds, stratus clouds and sea fog. Statistical analysis showed that the Rrc (412 nm) (Rayleigh Corrected Reflectance) of sea fog pixels is approximately 0.1-0.6. Similarly, various band combinations could be used to separate different surfaces. Therefore, three indices (SLDI, MCDI and BSI) were set to discern land/sea, middle/high level clouds and fog/stratus clouds, respectively, from which it was generally easy to extract fog pixels. The remote sensing algorithm was verified using coastal sounding data, which demonstrated that the algorithm had the ability to detect sea fog. The algorithm was then used to monitor an 8-hour sea fog event and the results were consistent with observational data from buoys data deployed near the Sheyang coast (121°E, 34°N). The goal of this study was to establish a daytime sea fog detection algorithm based on GOCI data, which shows promise for detecting fog separately from stratus.

Evaluating cloud retrieval algorithms with the ARM BBHRP framework

DOE Office of Scientific and Technical Information (OSTI.GOV)

Mlawer,E.; Dunn,M.; Mlawer, E.

2008-03-10

Climate and weather prediction models require accurate calculations of vertical profiles of radiative heating. Although heating rate calculations cannot be directly validated due to the lack of corresponding observations, surface and top-of-atmosphere measurements can indirectly establish the quality of computed heating rates through validation of the calculated irradiances at the atmospheric boundaries. The ARM Broadband Heating Rate Profile (BBHRP) project, a collaboration of all the working groups in the program, was designed with these heating rate validations as a key objective. Given the large dependence of radiative heating rates on cloud properties, a critical component of BBHRP radiative closure analysesmore » has been the evaluation of cloud microphysical retrieval algorithms. This evaluation is an important step in establishing the necessary confidence in the continuous profiles of computed radiative heating rates produced by BBHRP at the ARM Climate Research Facility (ACRF) sites that are needed for modeling studies. This poster details the continued effort to evaluate cloud property retrieval algorithms within the BBHRP framework, a key focus of the project this year. A requirement for the computation of accurate heating rate profiles is a robust cloud microphysical product that captures the occurrence, height, and phase of clouds above each ACRF site. Various approaches to retrieve the microphysical properties of liquid, ice, and mixed-phase clouds have been processed in BBHRP for the ACRF Southern Great Plains (SGP) and the North Slope of Alaska (NSA) sites. These retrieval methods span a range of assumptions concerning the parameterization of cloud location, particle density, size, shape, and involve different measurement sources. We will present the radiative closure results from several different retrieval approaches for the SGP site, including those from Microbase, the current 'reference' retrieval approach in BBHRP. At the NSA, mixed-phase clouds and cloud with a low optical depth are prevalent; the radiative closure studies using Microbase demonstrated significant residuals. As an alternative to Microbase at NSA, the Shupe-Turner cloud property retrieval algorithm, aimed at improving the partitioning of cloud phase and incorporating more constrained, conditional microphysics retrievals, also has been evaluated using the BBHRP data set.« less

Feature Relevance Assessment of Multispectral Airborne LIDAR Data for Tree Species Classification

NASA Astrophysics Data System (ADS)

Amiri, N.; Heurich, M.; Krzystek, P.; Skidmore, A. K.

2018-04-01

The presented experiment investigates the potential of Multispectral Laser Scanning (MLS) point clouds for single tree species classification. The basic idea is to simulate a MLS sensor by combining two different Lidar sensors providing three different wavelngthes. The available data were acquired in the summer 2016 at the same date in a leaf-on condition with an average point density of 37 points/m2. For the purpose of classification, we segmented the combined 3D point clouds consisiting of three different spectral channels into 3D clusters using Normalized Cut segmentation approach. Then, we extracted four group of features from the 3D point cloud space. Once a varity of features has been extracted, we applied forward stepwise feature selection in order to reduce the number of irrelevant or redundant features. For the classification, we used multinomial logestic regression with L1 regularization. Our study is conducted using 586 ground measured single trees from 20 sample plots in the Bavarian Forest National Park, in Germany. Due to lack of reference data for some rare species, we focused on four classes of species. The results show an improvement between 4-10 pp for the tree species classification by using MLS data in comparison to a single wavelength based approach. A cross validated (15-fold) accuracy of 0.75 can be achieved when all feature sets from three different spectral channels are used. Our results cleary indicates that the use of MLS point clouds has great potential to improve detailed forest species mapping.

Harmonic regression based multi-temporal cloud filtering algorithm for Landsat 8

NASA Astrophysics Data System (ADS)

Joshi, P.

2015-12-01

Landsat data archive though rich is seen to have missing dates and periods owing to the weather irregularities and inconsistent coverage. The satellite images are further subject to cloud cover effects resulting in erroneous analysis and observations of ground features. In earlier studies the change detection algorithm using statistical control charts on harmonic residuals of multi-temporal Landsat 5 data have been shown to detect few prominent remnant clouds [Brooks, Evan B., et al, 2014]. So, in this work we build on this harmonic regression approach to detect and filter clouds using a multi-temporal series of Landsat 8 images. Firstly, we compute the harmonic coefficients using the fitting models on annual training data. This time series of residuals is further subjected to Shewhart X-bar control charts which signal the deviations of cloud points from the fitted multi-temporal fourier curve. For the process with standard deviation σ we found the second and third order harmonic regression with a x-bar chart control limit [Lσ] ranging between [0.5σ < Lσ < σ] as most efficient in detecting clouds. By implementing second order harmonic regression with successive x-bar chart control limits of L and 0.5 L on the NDVI, NDSI and haze optimized transformation (HOT), and utilizing the seasonal physical properties of these parameters, we have designed a novel multi-temporal algorithm for filtering clouds from Landsat 8 images. The method is applied to Virginia and Alabama in Landsat8 UTM zones 17 and 16 respectively. Our algorithm efficiently filters all types of cloud cover with an overall accuracy greater than 90%. As a result of the multi-temporal operation and the ability to recreate the multi-temporal database of images using only the coefficients of the fourier regression, our algorithm is largely storage and time efficient. The results show a good potential for this multi-temporal approach for cloud detection as a timely and targeted solution for the Landsat 8 research community, catering to the need for innovative processing solutions in the infant stage of the satellite.

New Developments in the SCIAMACHY Level 2 Ground Processor Towards Version 7

NASA Astrophysics Data System (ADS)

Meringer, Markus; Noël, Stefan; Lichtenberg, Günter; Lerot, Christophe; Theys, Nicolas; Fehr, Thorsten; Dehn, Angelika; Liebing, Patricia; Gretschany, Sergei

2016-07-01

SCIAMACHY (SCanning Imaging Absorption spectroMeter for Atmospheric ChartographY) aboard ESA's environmental satellite ENVISAT observed the Earth's atmosphere in limb, nadir, and solar/lunar occultation geometries covering the UV-Visible to NIR spectral range. It is a joint project of Germany, the Netherlands and Belgium and was launched in February 2002. SCIAMACHY doubled its originally planned in-orbit lifetime of five years before the communication to ENVISAT was severed in April 2012, and the mission entered its post-operational phase. In order to preserve the best quality of the outstanding data recorded by SCIAMACHY, data processors are still being updated. This presentation will highlight three new developments that are currently being incorporated into the forthcoming version 7 of ESA's operational level 2 processor: 1. Tropospheric BrO, a new retrieval based on the scientific algorithm of (Theys et al., 2011). This algorithm had originally been developed for the GOME-2 sensor and was later adapted for SCIAMACHY. The main principle of the new algorithm is to split BrO total columns, which are already an operational product, into stratospheric VCD_{strat} and tropospheric VCD_{trop} fractions. BrO VCD_{strat} is determined from a climatological approach, driven by SCIAMACHY O_3 and NO_2 observations. Tropospheric vertical column densities are then determined as difference VCD_{trop}=VCD_{total}-VCD_{strat}. 2. Improved cloud flagging using limb measurements (Liebing, 2015). Limb cloud flags are already part of the SCIAMACHY L2 product. They are currently calculated employing the scientific algorithm developed by (Eichmann et al., 2015). Clouds are categorized into four types: water, ice, polar stratospheric and noctilucent clouds. High atmospheric aerosol loadings, however, often lead to spurious cloud flags, when aerosols had been misidentified as clouds. The new algorithm will better discriminate between aerosol and clouds. It will also have a higher sensitivity w.r.t. thin clouds. 3. A new, future-proof file format for the level 2 product based on NetCDF. The data format will be aligned and harmonized with other missions, particularly GOME and Sentinels. The final concept for the new format is still under discussion within the SCIAMACHY Quality Working Group. References: K.-U. Eichmann et al.: Global cloud top height retrieval using SCIAMACHY limb spectra: model studies and first results, Atmos. Meas. Tech. Discuss., 8, 8295-8352, 2015. P. Liebing: New Limb Cloud Detection Algorithm Theoretical Basis Document, 2016. N. Theys et al.: Global observations of tropospheric BrO columns using GOME-2 satellite data, Atmos. Chem. Phys., 11, 1791-1811, 2011.

Global Multispectral Cloud Retrievals from MODIS

NASA Technical Reports Server (NTRS)

King, Michael D.; Platnick, Steven; Ackerman, Steven A.; Menzel, W. Paul; Riedi, Jerome C.; Baum, Bryan A.

2003-01-01

The Moderate Resolution Imaging Spectroradiometer (MODIS) was developed by NASA and launched onboard the Terra spacecraft on December 18,1999 and Aqua spacecraft on May 4,2002. It achieved its final orbit and began Earth observations on February 24, 2000 for Terra and June 24, 2002 for Aqua. A comprehensive set of remote sensing algorithms for cloud masking and the retrieval of cloud physical and optical properties has been developed by members of the MODIS atmosphere science team. The archived products from these algorithms have applications in climate change studies, climate modeling, numerical weather prediction, as well as fundamental atmospheric research. In addition to an extensive cloud mask, products include cloud-top properties (temperature, pressure, effective emissivity), cloud thermodynamic phase, cloud optical and microphysical parameters (optical thickness, effective particle radius, water path), as well as derived statistics. We will describe the various cloud properties being analyzed on a global basis from both Terra and Aqua, and will show characteristics of cloud optical and microphysical properties as a function of latitude for land and ocean separately, and contrast the statistical properties of similar cloud types in various parts of the world.

Genetic Bee Colony (GBC) algorithm: A new gene selection method for microarray cancer classification.

PubMed

Alshamlan, Hala M; Badr, Ghada H; Alohali, Yousef A

2015-06-01

Naturally inspired evolutionary algorithms prove effectiveness when used for solving feature selection and classification problems. Artificial Bee Colony (ABC) is a relatively new swarm intelligence method. In this paper, we propose a new hybrid gene selection method, namely Genetic Bee Colony (GBC) algorithm. The proposed algorithm combines the used of a Genetic Algorithm (GA) along with Artificial Bee Colony (ABC) algorithm. The goal is to integrate the advantages of both algorithms. The proposed algorithm is applied to a microarray gene expression profile in order to select the most predictive and informative genes for cancer classification. In order to test the accuracy performance of the proposed algorithm, extensive experiments were conducted. Three binary microarray datasets are use, which include: colon, leukemia, and lung. In addition, another three multi-class microarray datasets are used, which are: SRBCT, lymphoma, and leukemia. Results of the GBC algorithm are compared with our recently proposed technique: mRMR when combined with the Artificial Bee Colony algorithm (mRMR-ABC). We also compared the combination of mRMR with GA (mRMR-GA) and Particle Swarm Optimization (mRMR-PSO) algorithms. In addition, we compared the GBC algorithm with other related algorithms that have been recently published in the literature, using all benchmark datasets. The GBC algorithm shows superior performance as it achieved the highest classification accuracy along with the lowest average number of selected genes. This proves that the GBC algorithm is a promising approach for solving the gene selection problem in both binary and multi-class cancer classification. Copyright © 2015 Elsevier Ltd. All rights reserved.

An Eight-Month Sample of Marine Stratocumulus Cloud Fraction, Albedo, and Integrated Liquid Water.

NASA Astrophysics Data System (ADS)

Fairall, C. W.; Hare, J. E.; Snider, J. B.

1990-08-01

As part of the First International Satellite Cloud Climatology Regional Experiment (FIRE), a surface meteorology and shortwave/longwave irradiance station was operated in a marine stratocumulus regime on the northwest tip of San Nicolas island off the coast of Southern California. Measurements were taken from March through October 1987, including a FIRE Intensive Field Operation (IFO) held in July. Algorithms were developed to use the longwave irradiance data to estimate fractional cloudiness and to use the shortwave irradiance to estimate cloud albedo and integrated cloud liquid water content. Cloud base height is estimated from computations of the lifting condensation level. The algorithms are tested against direct measurements made during the IFO; a 30% adjustment was made to the liquid water parameterization. The algorithms are then applied to the entire database. The stratocumulus clouds over the island are found to have a cloud base height of about 400 m, an integrated liquid water content of 75 gm2, a fractional cloudiness of 0.95, and an albedo of 0.55. Integrated liquid water content rarely exceeds 350 g m2 and albedo rarely exceeds 0.90 for stratocumulus clouds. Over the summer months, the average cloud fraction shows a maximum at sunrise of 0.74 and a minimum at sunset of 0.41. Over the same period, the average cloud albedo shows a maximum of 0.61 at sunrise and a minimum of 0.31 a few hours after local noon (although the estimate is more uncertain because of the extreme solar zenith angle). The use of joint frequency distributions of fractional cloudiness with solar transmittance or cloud base height to classify cloud types appears to be useful.

A novel artificial immune clonal selection classification and rule mining with swarm learning model

NASA Astrophysics Data System (ADS)

Al-Sheshtawi, Khaled A.; Abdul-Kader, Hatem M.; Elsisi, Ashraf B.

2013-06-01

Metaheuristic optimisation algorithms have become popular choice for solving complex problems. By integrating Artificial Immune clonal selection algorithm (CSA) and particle swarm optimisation (PSO) algorithm, a novel hybrid Clonal Selection Classification and Rule Mining with Swarm Learning Algorithm (CS2) is proposed. The main goal of the approach is to exploit and explore the parallel computation merit of Clonal Selection and the speed and self-organisation merits of Particle Swarm by sharing information between clonal selection population and particle swarm. Hence, we employed the advantages of PSO to improve the mutation mechanism of the artificial immune CSA and to mine classification rules within datasets. Consequently, our proposed algorithm required less training time and memory cells in comparison to other AIS algorithms. In this paper, classification rule mining has been modelled as a miltiobjective optimisation problem with predictive accuracy. The multiobjective approach is intended to allow the PSO algorithm to return an approximation to the accuracy and comprehensibility border, containing solutions that are spread across the border. We compared our proposed algorithm classification accuracy CS2 with five commonly used CSAs, namely: AIRS1, AIRS2, AIRS-Parallel, CLONALG, and CSCA using eight benchmark datasets. We also compared our proposed algorithm classification accuracy CS2 with other five methods, namely: Naïve Bayes, SVM, MLP, CART, and RFB. The results show that the proposed algorithm is comparable to the 10 studied algorithms. As a result, the hybridisation, built of CSA and PSO, can develop respective merit, compensate opponent defect, and make search-optimal effect and speed better.

A Locality-Constrained and Label Embedding Dictionary Learning Algorithm for Image Classification.

PubMed

Zhengming Li; Zhihui Lai; Yong Xu; Jian Yang; Zhang, David

2017-02-01

Locality and label information of training samples play an important role in image classification. However, previous dictionary learning algorithms do not take the locality and label information of atoms into account together in the learning process, and thus their performance is limited. In this paper, a discriminative dictionary learning algorithm, called the locality-constrained and label embedding dictionary learning (LCLE-DL) algorithm, was proposed for image classification. First, the locality information was preserved using the graph Laplacian matrix of the learned dictionary instead of the conventional one derived from the training samples. Then, the label embedding term was constructed using the label information of atoms instead of the classification error term, which contained discriminating information of the learned dictionary. The optimal coding coefficients derived by the locality-based and label-based reconstruction were effective for image classification. Experimental results demonstrated that the LCLE-DL algorithm can achieve better performance than some state-of-the-art algorithms.

[Combining speech sample and feature bilateral selection algorithm for classification of Parkinson's disease].

PubMed

Zhang, Xiaoheng; Wang, Lirui; Cao, Yao; Wang, Pin; Zhang, Cheng; Yang, Liuyang; Li, Yongming; Zhang, Yanling; Cheng, Oumei

2018-02-01

Diagnosis of Parkinson's disease (PD) based on speech data has been proved to be an effective way in recent years. However, current researches just care about the feature extraction and classifier design, and do not consider the instance selection. Former research by authors showed that the instance selection can lead to improvement on classification accuracy. However, no attention is paid on the relationship between speech sample and feature until now. Therefore, a new diagnosis algorithm of PD is proposed in this paper by simultaneously selecting speech sample and feature based on relevant feature weighting algorithm and multiple kernel method, so as to find their synergy effects, thereby improving classification accuracy. Experimental results showed that this proposed algorithm obtained apparent improvement on classification accuracy. It can obtain mean classification accuracy of 82.5%, which was 30.5% higher than the relevant algorithm. Besides, the proposed algorithm detected the synergy effects of speech sample and feature, which is valuable for speech marker extraction.

Morphological classification of odontogenic keratocysts using Bouligand-Minkowski fractal descriptors.

PubMed

Florindo, Joao B; Bruno, Odemir M; Landini, Gabriel

2017-02-01

The Odontogenic keratocyst (OKC) is a cystic lesion of the jaws, which has high growth and recurrence rates compared to other cysts of the jaws (for instance, radicular cyst, which is the most common jaw cyst type). For this reason OKCs are considered by some to be benign neoplasms. There exist two sub-types of OKCs (sporadic and syndromic) and the ability to discriminate between these sub-types, as well as other jaw cysts, is an important task in terms of disease diagnosis and prognosis. With the development of digital pathology, computational algorithms have become central to addressing this type of problem. Considering that only basic feature-based methods have been investigated in this problem before, we propose to use a different approach (the Bouligand-Minkowski descriptors) to assess the success rates achieved on the classification of a database of histological images of the epithelial lining of these cysts. This does not require the level of abstraction necessary to extract histologically-relevant features and therefore has the potential of being more robust than previous approaches. The descriptors were obtained by mapping pixel intensities into a three dimensional cloud of points in discrete space and applying morphological dilations with spheres of increasing radii. The descriptors were computed from the volume of the dilated set and submitted to a machine learning algorithm to classify the samples into diagnostic groups. This approach was capable of discriminating between OKCs and radicular cysts in 98% of images (100% of cases) and between the two sub-types of OKCs in 68% of images (71% of cases). These results improve over previously reported classification rates reported elsewhere and suggest that Bouligand-Minkowski descriptors are useful features to be used in histopathological images of these cysts. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

Estimates of Single Sensor Error Statistics for the MODIS Matchup Database Using Machine Learning

NASA Astrophysics Data System (ADS)

Kumar, C.; Podesta, G. P.; Minnett, P. J.; Kilpatrick, K. A.

2017-12-01

Sea surface temperature (SST) is a fundamental quantity for understanding weather and climate dynamics. Although sensors aboard satellites provide global and repeated SST coverage, a characterization of SST precision and bias is necessary for determining the suitability of SST retrievals in various applications. Guidance on how to derive meaningful error estimates is still being developed. Previous methods estimated retrieval uncertainty based on geophysical factors, e.g. season or "wet" and "dry" atmospheres, but the discrete nature of these bins led to spatial discontinuities in SST maps. Recently, a new approach clustered retrievals based on the terms (excluding offset) in the statistical algorithm used to estimate SST. This approach resulted in over 600 clusters - too many to understand the geophysical conditions that influence retrieval error. Using MODIS and buoy SST matchups (2002 - 2016), we use machine learning algorithms (recursive and conditional trees, random forests) to gain insight into geophysical conditions leading to the different signs and magnitudes of MODIS SST residuals (satellite SSTs minus buoy SSTs). MODIS retrievals were first split into three categories: < -0.4 C, -0.4 C ≤ residual ≤ 0.4 C, and > 0.4 C. These categories are heavily unbalanced, with residuals > 0.4 C being much less frequent. Performance of classification algorithms is affected by imbalance, thus we tested various rebalancing algorithms (oversampling, undersampling, combinations of the two). We consider multiple features for the decision tree algorithms: regressors from the MODIS SST algorithm, proxies for temperature deficit, and spatial homogeneity of brightness temperatures (BTs), e.g., the range of 11 μm BTs inside a 25 km2 area centered on the buoy location. These features and a rebalancing of classes led to an 81.9% accuracy when classifying SST retrievals into the < -0.4 C and -0.4 C ≤ residual ≤ 0.4 C categories. Spatial homogeneity in BTs consistently appears as a very important variable for classification, suggesting that unidentified cloud contamination still is one of the causes leading to negative SST residuals. Precision and accuracy of error estimates from our decision tree classifier are enhanced using this knowledge.

Remote Sensing of Radiative and Microphysical Properties of Clouds During TC (sup 4): Results from MAS, MASTER, MODIS, and MISR

NASA Technical Reports Server (NTRS)

King, Michael D.; Platnick, Steven; Wind, Galina; Arnold, G. Thomas; Dominguez, Roseanne T.

2010-01-01

The Moderate Resolution Imaging Spectroradiometer (MODIS) Airborne Simulator (MAS) and MODIS/Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) Airborne Simulator (MASTER) were used to obtain measurements of the bidirectional reflectance and brightness temperature of clouds at 50 discrete wavelengths between 0.47 and 14.2 microns (12.9 microns for MASTER). These observations were obtained from the NASA ER-2 aircraft as part of the Tropical Composition, Cloud and Climate Coupling (TC4) experiment conducted over Central America and surrounding Pacific and Atlantic Oceans between 17 July and 8 August 2007. Multispectral images in eleven distinct bands were used to derive a confidence in clear sky (or alternatively the probability Of cloud) over land and ocean ecosystems. Based on the results of individual tests run as part of the cloud mask, an algorithm was developed to estimate the phase of the clouds (liquid water, ice, or undetermined phase). The cloud optical thickness and effective radius were derived for both liquid water and ice clouds that were detected during each flight, using a nearly identical algorithm to that implemented operationally to process MODIS Cloud data from the Aqua and Terra satellites (Collection 5). This analysis shows that the cloud mask developed for operational use on MODIS, and tested using MAS and MASTER data in TC(sup 4), is quite capable of distinguishing both liquid water and ice clouds during daytime conditions over both land and ocean. The cloud optical thickness and effective radius retrievals use five distinct bands of the MAS (or MASTER), and these results were compared with nearly simultaneous retrievals of marine liquid water clouds from MODIS on the Terra spacecraft. Finally, this MODIS-based algorithm was adapted to Multiangle Imaging SpectroRadiometer (MISR) data to infer the cloud optical thickness Of liquid water clouds from MISR. Results of this analysis are compared and contrasted.

Progress towards MODIS and VIIRS Cloud Fraction Data Record Continuity

NASA Astrophysics Data System (ADS)

Ackerman, S. A.; Frey, R.; Holz, R.; Platnick, S. E.; Heidinger, A. K.

2016-12-01

Satellite-derived clear-sky vs. cloudy-sky discrimination at the pixel scale is an important input parameter used in many real-time applications. Cloud fractions, resulting from integrating over time and space, are also critical to the study of recent decadal climate changes. The NASA NPOESS Preparatory Project (NPP) has funded a science team to develop and study the ability to make continuous climate records from MODIS (2000-2020) and VIIRS (2012-2030). The MODAWG project, led by Dr. Steve Platnick of NASA/GSFC, combines elements of the MODIS processing system and the NOAA Algorithm Working Group (AWG) to achieve this goal. This presentation will focus on the cloud masking aspects of MODAWG, derived primarily from the MODIS cloud mask (MOD35). Challenges to continuity of cloud detection due to differences in instrument characteristics will be discussed. Cloud mask results from use of the same (continuity) algorithm will be shown for both MODIS and VIIRS, including comparisons to collocated CALIOP (Cloud-Aerosol Lidar with Orthogonal Polarization) cloud data.

Secure Scientific Applications Scheduling Technique for Cloud Computing Environment Using Global League Championship Algorithm

PubMed Central

Abdulhamid, Shafi’i Muhammad; Abd Latiff, Muhammad Shafie; Abdul-Salaam, Gaddafi; Hussain Madni, Syed Hamid

2016-01-01

Cloud computing system is a huge cluster of interconnected servers residing in a datacenter and dynamically provisioned to clients on-demand via a front-end interface. Scientific applications scheduling in the cloud computing environment is identified as NP-hard problem due to the dynamic nature of heterogeneous resources. Recently, a number of metaheuristics optimization schemes have been applied to address the challenges of applications scheduling in the cloud system, without much emphasis on the issue of secure global scheduling. In this paper, scientific applications scheduling techniques using the Global League Championship Algorithm (GBLCA) optimization technique is first presented for global task scheduling in the cloud environment. The experiment is carried out using CloudSim simulator. The experimental results show that, the proposed GBLCA technique produced remarkable performance improvement rate on the makespan that ranges between 14.44% to 46.41%. It also shows significant reduction in the time taken to securely schedule applications as parametrically measured in terms of the response time. In view of the experimental results, the proposed technique provides better-quality scheduling solution that is suitable for scientific applications task execution in the Cloud Computing environment than the MinMin, MaxMin, Genetic Algorithm (GA) and Ant Colony Optimization (ACO) scheduling techniques. PMID:27384239

Secure Scientific Applications Scheduling Technique for Cloud Computing Environment Using Global League Championship Algorithm.

PubMed

Abdulhamid, Shafi'i Muhammad; Abd Latiff, Muhammad Shafie; Abdul-Salaam, Gaddafi; Hussain Madni, Syed Hamid

2016-01-01

Cloud computing system is a huge cluster of interconnected servers residing in a datacenter and dynamically provisioned to clients on-demand via a front-end interface. Scientific applications scheduling in the cloud computing environment is identified as NP-hard problem due to the dynamic nature of heterogeneous resources. Recently, a number of metaheuristics optimization schemes have been applied to address the challenges of applications scheduling in the cloud system, without much emphasis on the issue of secure global scheduling. In this paper, scientific applications scheduling techniques using the Global League Championship Algorithm (GBLCA) optimization technique is first presented for global task scheduling in the cloud environment. The experiment is carried out using CloudSim simulator. The experimental results show that, the proposed GBLCA technique produced remarkable performance improvement rate on the makespan that ranges between 14.44% to 46.41%. It also shows significant reduction in the time taken to securely schedule applications as parametrically measured in terms of the response time. In view of the experimental results, the proposed technique provides better-quality scheduling solution that is suitable for scientific applications task execution in the Cloud Computing environment than the MinMin, MaxMin, Genetic Algorithm (GA) and Ant Colony Optimization (ACO) scheduling techniques.

A new tool for supervised classification of satellite images available on web servers: Google Maps as a case study

NASA Astrophysics Data System (ADS)

García-Flores, Agustín.; Paz-Gallardo, Abel; Plaza, Antonio; Li, Jun

2016-10-01

This paper describes a new web platform dedicated to the classification of satellite images called Hypergim. The current implementation of this platform enables users to perform classification of satellite images from any part of the world thanks to the worldwide maps provided by Google Maps. To perform this classification, Hypergim uses unsupervised algorithms like Isodata and K-means. Here, we present an extension of the original platform in which we adapt Hypergim in order to use supervised algorithms to improve the classification results. This involves a significant modification of the user interface, providing the user with a way to obtain samples of classes present in the images to use in the training phase of the classification process. Another main goal of this development is to improve the runtime of the image classification process. To achieve this goal, we use a parallel implementation of the Random Forest classification algorithm. This implementation is a modification of the well-known CURFIL software package. The use of this type of algorithms to perform image classification is widespread today thanks to its precision and ease of training. The actual implementation of Random Forest was developed using CUDA platform, which enables us to exploit the potential of several models of NVIDIA graphics processing units using them to execute general purpose computing tasks as image classification algorithms. As well as CUDA, we use other parallel libraries as Intel Boost, taking advantage of the multithreading capabilities of modern CPUs. To ensure the best possible results, the platform is deployed in a cluster of commodity graphics processing units (GPUs), so that multiple users can use the tool in a concurrent way. The experimental results indicate that this new algorithm widely outperform the previous unsupervised algorithms implemented in Hypergim, both in runtime as well as precision of the actual classification of the images.

CloudSat 2C-ICE product update with a new Ze parameterization in lidar-only region.

PubMed

Deng, Min; Mace, Gerald G; Wang, Zhien; Berry, Elizabeth

2015-12-16

The CloudSat 2C-ICE data product is derived from a synergetic ice cloud retrieval algorithm that takes as input a combination of CloudSat radar reflectivity ( Z e ) and Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation lidar attenuated backscatter profiles. The algorithm uses a variational method for retrieving profiles of visible extinction coefficient, ice water content, and ice particle effective radius in ice or mixed-phase clouds. Because of the nature of the measurements and to maintain consistency in the algorithm numerics, we choose to parameterize (with appropriately large specification of uncertainty) Z e and lidar attenuated backscatter in the regions of a cirrus layer where only the lidar provides data and where only the radar provides data, respectively. To improve the Z e parameterization in the lidar-only region, the relations among Z e , extinction, and temperature have been more thoroughly investigated using Atmospheric Radiation Measurement long-term millimeter cloud radar and Raman lidar measurements. This Z e parameterization provides a first-order estimation of Z e as a function extinction and temperature in the lidar-only regions of cirrus layers. The effects of this new parameterization have been evaluated for consistency using radiation closure methods where the radiative fluxes derived from retrieved cirrus profiles compare favorably with Clouds and the Earth's Radiant Energy System measurements. Results will be made publicly available for the entire CloudSat record (since 2006) in the most recent product release known as R05.

Improving Mixed-phase Cloud Parameterization in Climate Model with the ACRF Measurements

DOE Office of Scientific and Technical Information (OSTI.GOV)

Wang, Zhien

Mixed-phase cloud microphysical and dynamical processes are still poorly understood, and their representation in GCMs is a major source of uncertainties in overall cloud feedback in GCMs. Thus improving mixed-phase cloud parameterizations in climate models is critical to reducing the climate forecast uncertainties. This study aims at providing improved knowledge of mixed-phase cloud properties from the long-term ACRF observations and improving mixed-phase clouds simulations in the NCAR Community Atmosphere Model version 5 (CAM5). The key accomplishments are: 1) An improved retrieval algorithm was developed to provide liquid droplet concentration for drizzling or mixed-phase stratiform clouds. 2) A new ice concentrationmore » retrieval algorithm for stratiform mixed-phase clouds was developed. 3) A strong seasonal aerosol impact on ice generation in Arctic mixed-phase clouds was identified, which is mainly attributed to the high dust occurrence during the spring season. 4) A suite of multi-senor algorithms was applied to long-term ARM observations at the Barrow site to provide a complete dataset (LWC and effective radius profile for liquid phase, and IWC, Dge profiles and ice concentration for ice phase) to characterize Arctic stratiform mixed-phase clouds. This multi-year stratiform mixed-phase cloud dataset provides necessary information to study related processes, evaluate model stratiform mixed-phase cloud simulations, and improve model stratiform mixed-phase cloud parameterization. 5). A new in situ data analysis method was developed to quantify liquid mass partition in convective mixed-phase clouds. For the first time, we reliably compared liquid mass partitions in stratiform and convective mixed-phase clouds. Due to the different dynamics in stratiform and convective mixed-phase clouds, the temperature dependencies of liquid mass partitions are significantly different due to much higher ice concentrations in convective mixed phase clouds. 6) Systematic evaluations of mixed-phase cloud simulations by CAM5 were performed. Measurement results indicate that ice concentrations control stratiform mixed-phase cloud properties. The improvement of ice concentration parameterization in the CAM5 was done in close collaboration with Dr. Xiaohong Liu, PNNL (now at University of Wyoming).« less

Workflow as a Service in the Cloud: Architecture and Scheduling Algorithms.

PubMed

Wang, Jianwu; Korambath, Prakashan; Altintas, Ilkay; Davis, Jim; Crawl, Daniel

2014-01-01

With more and more workflow systems adopting cloud as their execution environment, it becomes increasingly challenging on how to efficiently manage various workflows, virtual machines (VMs) and workflow execution on VM instances. To make the system scalable and easy-to-extend, we design a Workflow as a Service (WFaaS) architecture with independent services. A core part of the architecture is how to efficiently respond continuous workflow requests from users and schedule their executions in the cloud. Based on different targets, we propose four heuristic workflow scheduling algorithms for the WFaaS architecture, and analyze the differences and best usages of the algorithms in terms of performance, cost and the price/performance ratio via experimental studies.

Added value of far-infrared radiometry for remote sensing of ice clouds

NASA Astrophysics Data System (ADS)

Libois, Quentin; Blanchet, Jean-Pierre

2017-06-01

Several cloud retrieval algorithms based on satellite observations in the infrared have been developed in the last decades. However, these observations only cover the midinfrared (MIR, λ < 15 μm) part of the spectrum, and none are available in the far-infrared (FIR, λ≥ 15 μm). Using the optimal estimation method, we show that adding a few FIR channels to existing spaceborne radiometers would significantly improve their ability to retrieve ice cloud radiative properties. For clouds encountered in the polar regions and the upper troposphere, where the atmosphere is sufficiently transparent in the FIR, using FIR channels would reduce by more than 50% the uncertainties on retrieved values of optical thickness, effective particle diameter, and cloud top altitude. Notably, this would extend the range of applicability of current retrieval methods to the polar regions and to clouds with large optical thickness, where MIR algorithms perform poorly. The high performance of solar reflection-based algorithms would thus be reached in nighttime conditions. Since the sensitivity of ice cloud thermal emission to effective particle diameter is approximately 5 times larger in the FIR than in the MIR, using FIR observations is a promising venue for studying ice cloud microphysics and precipitation processes. This is highly relevant for cirrus clouds and convective towers. This is also essential to study precipitation in the driest regions of the atmosphere, where strong feedbacks are at play between clouds and water vapor. The deployment in the near future of a FIR spaceborne radiometer is technologically feasible and should be strongly supported.

Efficient Fingercode Classification

NASA Astrophysics Data System (ADS)

Sun, Hong-Wei; Law, Kwok-Yan; Gollmann, Dieter; Chung, Siu-Leung; Li, Jian-Bin; Sun, Jia-Guang

In this paper, we present an efficient fingerprint classification algorithm which is an essential component in many critical security application systems e. g. systems in the e-government and e-finance domains. Fingerprint identification is one of the most important security requirements in homeland security systems such as personnel screening and anti-money laundering. The problem of fingerprint identification involves searching (matching) the fingerprint of a person against each of the fingerprints of all registered persons. To enhance performance and reliability, a common approach is to reduce the search space by firstly classifying the fingerprints and then performing the search in the respective class. Jain et al. proposed a fingerprint classification algorithm based on a two-stage classifier, which uses a K-nearest neighbor classifier in its first stage. The fingerprint classification algorithm is based on the fingercode representation which is an encoding of fingerprints that has been demonstrated to be an effective fingerprint biometric scheme because of its ability to capture both local and global details in a fingerprint image. We enhance this approach by improving the efficiency of the K-nearest neighbor classifier for fingercode-based fingerprint classification. Our research firstly investigates the various fast search algorithms in vector quantization (VQ) and the potential application in fingerprint classification, and then proposes two efficient algorithms based on the pyramid-based search algorithms in VQ. Experimental results on DB1 of FVC 2004 demonstrate that our algorithms can outperform the full search algorithm and the original pyramid-based search algorithms in terms of computational efficiency without sacrificing accuracy.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Martin, Shawn

This code consists of Matlab routines which enable the user to perform non-manifold surface reconstruction via triangulation from high dimensional point cloud data. The code was based on an algorithm originally developed in [Freedman (2007), An Incremental Algorithm for Reconstruction of Surfaces of Arbitrary Codimension Computational Geometry: Theory and Applications, 36(2):106-116]. This algorithm has been modified to accommodate non-manifold surface according to the work described in [S. Martin and J.-P. Watson (2009), Non-Manifold Surface Reconstruction from High Dimensional Point Cloud DataSAND #5272610].The motivation for developing the code was a point cloud describing the molecular conformation space of cyclooctane (C8H16). Cyclooctanemore » conformation space was represented using points in 72 dimensions (3 coordinates for each molecule). The code was used to triangulate the point cloud and thereby study the geometry and topology of cyclooctane. Futures applications are envisioned for peptides and proteins.« less

Outdoor Illegal Construction Identification Algorithm Based on 3D Point Cloud Segmentation

NASA Astrophysics Data System (ADS)

An, Lu; Guo, Baolong

2018-03-01

Recently, various illegal constructions occur significantly in our surroundings, which seriously restrict the orderly development of urban modernization. The 3D point cloud data technology is used to identify the illegal buildings, which could address the problem above effectively. This paper proposes an outdoor illegal construction identification algorithm based on 3D point cloud segmentation. Initially, in order to save memory space and reduce processing time, a lossless point cloud compression method based on minimum spanning tree is proposed. Then, a ground point removing method based on the multi-scale filtering is introduced to increase accuracy. Finally, building clusters on the ground can be obtained using a region growing method, as a result, the illegal construction can be marked. The effectiveness of the proposed algorithm is verified using a publicly data set collected from the International Society for Photogrammetry and Remote Sensing (ISPRS).

Infrared remote sensing of the vertical and horizontal distribution of clouds

NASA Technical Reports Server (NTRS)

Chahine, M. T.; Haskins, R. D.

1982-01-01

An algorithm has been developed to derive the horizontal and vertical distribution of clouds from the same set of infrared radiance data used to retrieve atmospheric temperature profiles. The method leads to the determination of the vertical atmospheric temperature structure and the cloud distribution simultaneously, providing information on heat sources and sinks, storage rates and transport phenomena in the atmosphere. Experimental verification of this algorithm was obtained using the 15-micron data measured by the NOAA-VTPR temperature sounder. After correcting for water vapor emission, the results show that the cloud cover derived from 15-micron data is less than that obtained from visible data.

A model of cloud application assignments in software-defined storages

NASA Astrophysics Data System (ADS)

Bolodurina, Irina P.; Parfenov, Denis I.; Polezhaev, Petr N.; E Shukhman, Alexander

2017-01-01

The aim of this study is to analyze the structure and mechanisms of interaction of typical cloud applications and to suggest the approaches to optimize their placement in storage systems. In this paper, we describe a generalized model of cloud applications including the three basic layers: a model of application, a model of service, and a model of resource. The distinctive feature of the model suggested implies analyzing cloud resources from the user point of view and from the point of view of a software-defined infrastructure of the virtual data center (DC). The innovation character of this model is in describing at the same time the application data placements, as well as the state of the virtual environment, taking into account the network topology. The model of software-defined storage has been developed as a submodel within the resource model. This model allows implementing the algorithm for control of cloud application assignments in software-defined storages. Experimental researches returned this algorithm decreases in cloud application response time and performance growth in user request processes. The use of software-defined data storages allows the decrease in the number of physical store devices, which demonstrates the efficiency of our algorithm.

Supporting reputation based trust management enhancing security layer for cloud service models

NASA Astrophysics Data System (ADS)

Karthiga, R.; Vanitha, M.; Sumaiya Thaseen, I.; Mangaiyarkarasi, R.

2017-11-01

In the existing system trust between cloud providers and consumers is inadequate to establish the service level agreement though the consumer’s response is good cause to assess the overall reliability of cloud services. Investigators recognized the significance of trust can be managed and security can be provided based on feedback collected from participant. In this work a face recognition system that helps to identify the user effectively. So we use an image comparison algorithm where the user face is captured during registration time and get stored in database. With that original image we compare it with the sample image that is already stored in database. If both the image get matched then the users are identified effectively. When the confidential data are subcontracted to the cloud, data holders will become worried about the confidentiality of their data in the cloud. Encrypting the data before subcontracting has been regarded as the important resources of keeping user data privacy beside the cloud server. So in order to keep the data secure we use an AES algorithm. Symmetric-key algorithms practice a shared key concept, keeping data secret requires keeping this key secret. So only the user with private key can decrypt data.

PROCAMS - A second generation multispectral-multitemporal data processing system for agricultural mensuration

NASA Technical Reports Server (NTRS)

Erickson, J. D.; Nalepka, R. F.

1976-01-01

PROCAMS (Prototype Classification and Mensuration System) has been designed for the classification and mensuration of agricultural crops (specifically small grains including wheat, rye, oats, and barley) through the use of data provided by Landsat. The system includes signature extension as a major feature and incorporates multitemporal as well as early season unitemporal approaches for using multiple training sites. Also addressed are partial cloud cover and cloud shadows, bad data points and lines, as well as changing sun angle and atmospheric state variations.

Shadow detection and removal in RGB VHR images for land use unsupervised classification

NASA Astrophysics Data System (ADS)

Movia, A.; Beinat, A.; Crosilla, F.

2016-09-01

Nowadays, high resolution aerial images are widely available thanks to the diffusion of advanced technologies such as UAVs (Unmanned Aerial Vehicles) and new satellite missions. Although these developments offer new opportunities for accurate land use analysis and change detection, cloud and terrain shadows actually limit benefits and possibilities of modern sensors. Focusing on the problem of shadow detection and removal in VHR color images, the paper proposes new solutions and analyses how they can enhance common unsupervised classification procedures for identifying land use classes related to the CO2 absorption. To this aim, an improved fully automatic procedure has been developed for detecting image shadows using exclusively RGB color information, and avoiding user interaction. Results show a significant accuracy enhancement with respect to similar methods using RGB based indexes. Furthermore, novel solutions derived from Procrustes analysis have been applied to remove shadows and restore brightness in the images. In particular, two methods implementing the so called "anisotropic Procrustes" and the "not-centered oblique Procrustes" algorithms have been developed and compared with the linear correlation correction method based on the Cholesky decomposition. To assess how shadow removal can enhance unsupervised classifications, results obtained with classical methods such as k-means, maximum likelihood, and self-organizing maps, have been compared to each other and with a supervised clustering procedure.

THE USE OF BOX MODELS TO DESCRIBE THE PERSONAL CLOUD EFFECT ON HUMAN EXPOSURE TO PARTICULATE MATTER

EPA Science Inventory

An algorithm has been developed to describe particle transport into and out of the breathing zone in an effort to predict the effects of the personal cloud phenomenon (Eisner and Heist, 2000). The algorithm was developed based on the principle of mass balance between a system ...

Sparse Unorganized Point Cloud Based Relative Pose Estimation for Uncooperative Space Target.

PubMed

Yin, Fang; Chou, Wusheng; Wu, Yun; Yang, Guang; Xu, Song

2018-03-28

This paper proposes an autonomous algorithm to determine the relative pose between the chaser spacecraft and the uncooperative space target, which is essential in advanced space applications, e.g., on-orbit serving missions. The proposed method, named Congruent Tetrahedron Align (CTA) algorithm, uses the very sparse unorganized 3D point cloud acquired by a LIDAR sensor, and does not require any prior pose information. The core of the method is to determine the relative pose by looking for the congruent tetrahedron in scanning point cloud and model point cloud on the basis of its known model. The two-level index hash table is built for speeding up the search speed. In addition, the Iterative Closest Point (ICP) algorithm is used for pose tracking after CTA. In order to evaluate the method in arbitrary initial attitude, a simulated system is presented. Specifically, the performance of the proposed method to provide the initial pose needed for the tracking algorithm is demonstrated, as well as their robustness against noise. Finally, a field experiment is conducted and the results demonstrated the effectiveness of the proposed method.

Day/night whole sky imagers for 24-h cloud and sky assessment: history and overview.

PubMed

Shields, Janet E; Karr, Monette E; Johnson, Richard W; Burden, Art R

2013-03-10

A family of fully automated digital whole sky imagers (WSIs) has been developed at the Marine Physical Laboratory over many years, for a variety of research and military applications. The most advanced of these, the day/night whole sky imagers (D/N WSIs), acquire digital imagery of the full sky down to the horizon under all conditions from full sunlight to starlight. Cloud algorithms process the imagery to automatically detect the locations of cloud for both day and night. The instruments can provide absolute radiance distribution over the full radiance range from starlight through daylight. The WSIs were fielded in 1984, followed by the D/N WSIs in 1992. These many years of experience and development have resulted in very capable instruments and algorithms that remain unique. This article discusses the history of the development of the D/N WSIs, system design, algorithms, and data products. The paper cites many reports with more detailed technical documentation. Further details of calibration, day and night algorithms, and cloud free line-of-sight results will be discussed in future articles.

Sparse Unorganized Point Cloud Based Relative Pose Estimation for Uncooperative Space Target

PubMed Central

Chou, Wusheng; Wu, Yun; Yang, Guang; Xu, Song

2018-01-01

This paper proposes an autonomous algorithm to determine the relative pose between the chaser spacecraft and the uncooperative space target, which is essential in advanced space applications, e.g., on-orbit serving missions. The proposed method, named Congruent Tetrahedron Align (CTA) algorithm, uses the very sparse unorganized 3D point cloud acquired by a LIDAR sensor, and does not require any prior pose information. The core of the method is to determine the relative pose by looking for the congruent tetrahedron in scanning point cloud and model point cloud on the basis of its known model. The two-level index hash table is built for speeding up the search speed. In addition, the Iterative Closest Point (ICP) algorithm is used for pose tracking after CTA. In order to evaluate the method in arbitrary initial attitude, a simulated system is presented. Specifically, the performance of the proposed method to provide the initial pose needed for the tracking algorithm is demonstrated, as well as their robustness against noise. Finally, a field experiment is conducted and the results demonstrated the effectiveness of the proposed method. PMID:29597323

Spatial and Temporal Distribution of Tropospheric Clouds Observed by MODIS Onboard the Terra and Aqua Satellites

NASA Technical Reports Server (NTRS)

King, Michael D.

2005-01-01

The Moderate Resolution Imaging Spectroradiometer (MODIS) was developed by NASA and launched onboard the Terra spacecraft on December 18, 1999 and Aqua spacecraft on May 4, 2002. It achieved its final orbit and began Earth observations on February 24, 2000 for Terra and June 24, 2002 for Aqua. A comprehensive set of remote sensing algorithms for cloud masking and the retrieval of cloud physical and optical properties has been developed by members of the MODIS atmosphere science team. The archived products from these algorithms have applications in climate change studies, climate modeling, numerical weather prediction, as well as fundamental atmospheric research. In addition to an extensive cloud mask, products include cloud-top properties (temperature, pressure, effective emissivity), cloud thermodynamic phase, cloud optical and microphysical parameters (optical thickness, effective particle radius, water path), as well as derived statistics. We will describe the various cloud properties being analyzed on a global basis from both Terra and Aqua. These include the latitudinal distribution of cloud optical and radiative properties of both liquid water and ice clouds, as well as joint histograms of cloud optical thickness and effective radius for selected geographical locations around the world.

Spatial and Temporal Distribution of Tropospheric Clouds Observed by MODIS Onboard the Terra and Aqua Satellites

NASA Technical Reports Server (NTRS)

King, Michael D.; Platnick, Steven

2005-01-01

The Moderate Resolution Imaging Spectroradiometer (MODIS) was developed by NASA and launched onboard the Terra spacecraft on December 18,1999 and Aqua spacecraft on May 4, 2002. It achieved its final orbit and began Earth observations on February 24, 2000 for Terra and June 24, 2002 for Aqua. A comprehensive set of remote sensing algorithms for cloud masking and the retrieval of cloud physical and optical properties has been developed by members of the MODIS atmosphere science team. The archived products from these algorithms have applications in climate change studies, climate modeling, numerical weather prediction, as well as fundamental atmospheric research. In addition to an extensive cloud mask, products include cloud-top properties (temperature, pressure, effective emissivity), cloud thermodynamic phase, cloud optical and microphysical parameters (optical thickness, effective particle radius, water path), as well as derived statistics. We will describe the various cloud properties being analyzed on a global basis from both Terra and Aqua. These include the latitudinal distribution of cloud optical and radiative properties of both liquid water and ice clouds, as well as joint histograms of cloud optical thickness and effective radius for selected geographical locations around the world.

Online clustering algorithms for radar emitter classification.

PubMed

Liu, Jun; Lee, Jim P Y; Senior; Li, Lingjie; Luo, Zhi-Quan; Wong, K Max

2005-08-01

Radar emitter classification is a special application of data clustering for classifying unknown radar emitters from received radar pulse samples. The main challenges of this task are the high dimensionality of radar pulse samples, small sample group size, and closely located radar pulse clusters. In this paper, two new online clustering algorithms are developed for radar emitter classification: One is model-based using the Minimum Description Length (MDL) criterion and the other is based on competitive learning. Computational complexity is analyzed for each algorithm and then compared. Simulation results show the superior performance of the model-based algorithm over competitive learning in terms of better classification accuracy, flexibility, and stability.

Spectral band selection for classification of soil organic matter content

NASA Technical Reports Server (NTRS)

Henderson, Tracey L.; Szilagyi, Andrea; Baumgardner, Marion F.; Chen, Chih-Chien Thomas; Landgrebe, David A.

1989-01-01

This paper describes the spectral-band-selection (SBS) algorithm of Chen and Landgrebe (1987, 1988, and 1989) and uses the algorithm to classify the organic matter content in the earth's surface soil. The effectiveness of the algorithm was evaluated comparing the results of classification of the soil organic matter using SBS bands with those obtained using Landsat MSS bands and TM bands, showing that the algorithm was successful in finding important spectral bands for classification of organic matter content. Using the calculated bands, the probabilities of correct classification for climate-stratified data were found to range from 0.910 to 0.980.

Retrieval of Ice Cloud Properties Using an Optimal Estimation Algorithm and MODIS Infrared Observations: 2. Retrieval Evaluation

NASA Technical Reports Server (NTRS)

Wang, Chenxi; Platnick, Steven; Zhang, Zhibo; Meyer, Kerry; Wind, Galina; Yang, Ping

2016-01-01

An infrared-based optimal estimation (OE-IR) algorithm for retrieving ice cloud properties is evaluated. Specifically, the implementation of the algorithm with MODerate resolution Imaging Spectroradiometer (MODIS) observations is assessed in comparison with the operational retrieval products from MODIS on the Aqua satellite (MYD06), Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP), and the Imaging Infrared Radiometer (IIR); the latter two instruments fly on the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) satellite in the Afternoon Constellation (A-Train) with Aqua. The results show that OE-IR cloud optical thickness (tau) and effective radius (r(sub eff)) retrievals perform best for ice clouds having 0.5 < tau< 7 and r(sub eff) < 50microns. For global ice clouds, the averaged retrieval uncertainties of tau and r(sub eff) are 19% and 33%, respectively. For optically thick ice clouds with tau larger than 10, however, the tau and r(sub eff) retrieval uncertainties can exceed 30% and 50%, respectively. For ice cloud top height (h), the averaged global uncertainty is 0.48km. Relatively large h uncertainty (e.g., > 1km) occurs for tau < 0.5. Analysis of 1month of the OE-IR retrievals shows large tau and r(sub eff) uncertainties in storm track regions and the southern oceans where convective clouds are frequently observed, as well as in high-latitude regions where temperature differences between the surface and cloud top are more ambiguous. Generally, comparisons between the OE-IR and the operational products show consistent tau and h retrievals. However, obvious differences between the OE-IR and the MODIS Collection 6 r(sub eff) are found.

Multi-label classification of chronically ill patients with bag of words and supervised dimensionality reduction algorithms.

PubMed

Bromuri, Stefano; Zufferey, Damien; Hennebert, Jean; Schumacher, Michael

2014-10-01

This research is motivated by the issue of classifying illnesses of chronically ill patients for decision support in clinical settings. Our main objective is to propose multi-label classification of multivariate time series contained in medical records of chronically ill patients, by means of quantization methods, such as bag of words (BoW), and multi-label classification algorithms. Our second objective is to compare supervised dimensionality reduction techniques to state-of-the-art multi-label classification algorithms. The hypothesis is that kernel methods and locality preserving projections make such algorithms good candidates to study multi-label medical time series. We combine BoW and supervised dimensionality reduction algorithms to perform multi-label classification on health records of chronically ill patients. The considered algorithms are compared with state-of-the-art multi-label classifiers in two real world datasets. Portavita dataset contains 525 diabetes type 2 (DT2) patients, with co-morbidities of DT2 such as hypertension, dyslipidemia, and microvascular or macrovascular issues. MIMIC II dataset contains 2635 patients affected by thyroid disease, diabetes mellitus, lipoid metabolism disease, fluid electrolyte disease, hypertensive disease, thrombosis, hypotension, chronic obstructive pulmonary disease (COPD), liver disease and kidney disease. The algorithms are evaluated using multi-label evaluation metrics such as hamming loss, one error, coverage, ranking loss, and average precision. Non-linear dimensionality reduction approaches behave well on medical time series quantized using the BoW algorithm, with results comparable to state-of-the-art multi-label classification algorithms. Chaining the projected features has a positive impact on the performance of the algorithm with respect to pure binary relevance approaches. The evaluation highlights the feasibility of representing medical health records using the BoW for multi-label classification tasks. The study also highlights that dimensionality reduction algorithms based on kernel methods, locality preserving projections or both are good candidates to deal with multi-label classification tasks in medical time series with many missing values and high label density. Copyright © 2014 Elsevier Inc. All rights reserved.

Coastal Algorithms and On-Demand Processing- The Lessons Learnt from CoastColour for Sentinel 3

NASA Astrophysics Data System (ADS)

Brockmann, Carsten; Doerffer, Roland; Boettcher, Martin; Kramer, Uwe; Zuhlke, Marco; Pinnock, Simon

2015-12-01

The ESA DUE CoastColour Project has been initiated to provide water quality products for important costal zones globally. A new 5 component bio-optical model was developed and used in a 3-step approach for regional processing of ocean colour data. The L1P step consists of radiometric and geometric system corrections, and top-of-atmosphere pixel classification including cloud screening, sun glint risk masking or detection of floating vegetation. The second step includes the atmospheric correction and is providing the L2R product, which comprises marine reflectances with error characterisation and normalisation. The third step is the in-water processing which produces IOPs, attenuation coefficient and water constituent concentrations. Each of these steps will benefit from the additional bands on OLCI. The 5 component bio-optical model will already be used in the standard ESA processing of OLCI, and also part of the pixel classification methods will be part of the standard products. Other algorithm adaptation are in preparation. Another important advantage of the CoastColour approach is the highly configurable processing chain which allows adaptation to the individual characteristics of the area of interest, temporal window, algorithm parametrisation and processing chain configuration. This flexibility is made available to data users through the CoastColour on-demand processing service. The complete global MERIS Full and Reduced Resolution data archive is accessible, covering the time range from 17. May 2002 until 08. April 2012, which is almost 200TB of in-put data available online. The CoastColour on-demand processing service can serve as a model for hosted processing, where the software is moved to the data instead of moving the data to the users, which will be a challenge with the large amount of data coming from Sentinel 3.

Fusion of optical and SAR remote sensing images for tropical forests monitoring

NASA Astrophysics Data System (ADS)

Wang, C.; Yu, M.; Gao, Q.; Wang, X.

2016-12-01

Although tropical deforestation prevails in South America and Southeast Asia, reforestation appeared in some tropical regions due to economic changes. After the economic shift from agriculture to industry, the tropical island of Puerto Rico has experienced rapid reforestation as well as urban expansion since the late 1940s. Continued urban growth without the guide of sustainable planning might prevent further forest regrowth. Accurate and timely mapping of LULC is of great importance for evaluating the consequences of reforestation and urban expansion on the coupled human and nature systems. However, owning to persistent cloud cover in tropics, it remains a challenge to produce reliable LULC maps in fine spatial resolution. Here, we retrieved cloud-free Landsat surface reflectance composite data by removing clouds and shades from the USGS Landsat Surface Reflectance (SR) product for each scene using the CFmask and Fmask algorithms in Google Earth Engine. We then produced high accuracy land cover classification maps using SR optical data for the year of 2000 and fused optical and ALOS SAR data for 2010 and 2015, with an overall accuracy of 92.0%, 92.5%, and 91.6%, respectively. The classification result indicated that a successive forest gain of 6.52% and 1.03% occurred between the first (2000-2010) and second (2010-2015) study periods, respectively. We also conducted a comparative spatial analysis of patterns of deforestation and reforestation based on a series of forest cover zones (50 × 50 pixels, 150 ha). The annual rates of deforestation and reforestation against forest cover presented the similar trends during two periods: decreasing with the forest cover increasing. However, the annual net forest change rate was different in the zones with forest cover less than 30%, presenting significant gain (2.2-8.4% yr-1) for the first period and significant loss (2.3-6.4% yr-1) for the second period. It indicated that both deforestation and reforestation mostly occurred near the forest edges and low density secondary forests.

Cloud Computing: A model Construct of Real-Time Monitoring for Big Dataset Analytics Using Apache Spark

NASA Astrophysics Data System (ADS)

Alkasem, Ameen; Liu, Hongwei; Zuo, Decheng; Algarash, Basheer

2018-01-01

The volume of data being collected, analyzed, and stored has exploded in recent years, in particular in relation to the activity on the cloud computing. While large-scale data processing, analysis, storage, and platform model such as cloud computing were previously and currently are increasingly. Today, the major challenge is it address how to monitor and control these massive amounts of data and perform analysis in real-time at scale. The traditional methods and model systems are unable to cope with these quantities of data in real-time. Here we present a new methodology for constructing a model for optimizing the performance of real-time monitoring of big datasets, which includes a machine learning algorithms and Apache Spark Streaming to accomplish fine-grained fault diagnosis and repair of big dataset. As a case study, we use the failure of Virtual Machines (VMs) to start-up. The methodology proposition ensures that the most sensible action is carried out during the procedure of fine-grained monitoring and generates the highest efficacy and cost-saving fault repair through three construction control steps: (I) data collection; (II) analysis engine and (III) decision engine. We found that running this novel methodology can save a considerate amount of time compared to the Hadoop model, without sacrificing the classification accuracy or optimization of performance. The accuracy of the proposed method (92.13%) is an improvement on traditional approaches.

Advances in simultaneous atmospheric profile and cloud parameter regression based retrieval from high-spectral resolution radiance measurements

NASA Astrophysics Data System (ADS)

Weisz, Elisabeth; Smith, William L.; Smith, Nadia

2013-06-01

The dual-regression (DR) method retrieves information about the Earth surface and vertical atmospheric conditions from measurements made by any high-spectral resolution infrared sounder in space. The retrieved information includes temperature and atmospheric gases (such as water vapor, ozone, and carbon species) as well as surface and cloud top parameters. The algorithm was designed to produce a high-quality product with low latency and has been demonstrated to yield accurate results in real-time environments. The speed of the retrieval is achieved through linear regression, while accuracy is achieved through a series of classification schemes and decision-making steps. These steps are necessary to account for the nonlinearity of hyperspectral retrievals. In this work, we detail the key steps that have been developed in the DR method to advance accuracy in the retrieval of nonlinear parameters, specifically cloud top pressure. The steps and their impact on retrieval results are discussed in-depth and illustrated through relevant case studies. In addition to discussing and demonstrating advances made in addressing nonlinearity in a linear geophysical retrieval method, advances toward multi-instrument geophysical analysis by applying the DR to three different operational sounders in polar orbit are also noted. For any area on the globe, the DR method achieves consistent accuracy and precision, making it potentially very valuable to both the meteorological and environmental user communities.

An adaptive surface filter for airborne laser scanning point clouds by means of regularization and bending energy

NASA Astrophysics Data System (ADS)

Hu, Han; Ding, Yulin; Zhu, Qing; Wu, Bo; Lin, Hui; Du, Zhiqiang; Zhang, Yeting; Zhang, Yunsheng

2014-06-01

The filtering of point clouds is a ubiquitous task in the processing of airborne laser scanning (ALS) data; however, such filtering processes are difficult because of the complex configuration of the terrain features. The classical filtering algorithms rely on the cautious tuning of parameters to handle various landforms. To address the challenge posed by the bundling of different terrain features into a single dataset and to surmount the sensitivity of the parameters, in this study, we propose an adaptive surface filter (ASF) for the classification of ALS point clouds. Based on the principle that the threshold should vary in accordance to the terrain smoothness, the ASF embeds bending energy, which quantitatively depicts the local terrain structure to self-adapt the filter threshold automatically. The ASF employs a step factor to control the data pyramid scheme in which the processing window sizes are reduced progressively, and the ASF gradually interpolates thin plate spline surfaces toward the ground with regularization to handle noise. Using the progressive densification strategy, regularization and self-adaption, both performance improvement and resilience to parameter tuning are achieved. When tested against the benchmark datasets provided by ISPRS, the ASF performs the best in comparison with all other filtering methods, yielding an average total error of 2.85% when optimized and 3.67% when using the same parameter set.

Improved ocean-color remote sensing in the Arctic using the POLYMER algorithm

NASA Astrophysics Data System (ADS)

Frouin, Robert; Deschamps, Pierre-Yves; Ramon, Didier; Steinmetz, François

2012-10-01

Atmospheric correction of ocean-color imagery in the Arctic brings some specific challenges that the standard atmospheric correction algorithm does not address, namely low solar elevation, high cloud frequency, multi-layered polar clouds, presence of ice in the field-of-view, and adjacency effects from highly reflecting surfaces covered by snow and ice and from clouds. The challenges may be addressed using a flexible atmospheric correction algorithm, referred to as POLYMER (Steinmetz and al., 2011). This algorithm does not use a specific aerosol model, but fits the atmospheric reflectance by a polynomial with a non spectral term that accounts for any non spectral scattering (clouds, coarse aerosol mode) or reflection (glitter, whitecaps, small ice surfaces within the instrument field of view), a spectral term with a law in wavelength to the power -1 (fine aerosol mode), and a spectral term with a law in wavelength to the power -4 (molecular scattering, adjacency effects from clouds and white surfaces). Tests are performed on selected MERIS imagery acquired over Arctic Seas. The derived ocean properties, i.e., marine reflectance and chlorophyll concentration, are compared with those obtained with the standard MEGS algorithm. The POLYMER estimates are more realistic in regions affected by the ice environment, e.g., chlorophyll concentration is higher near the ice edge, and spatial coverage is substantially increased. Good retrievals are obtained in the presence of thin clouds, with ocean-color features exhibiting spatial continuity from clear to cloudy regions. The POLYMER estimates of marine reflectance agree better with in situ measurements than the MEGS estimates. Biases are 0.001 or less in magnitude, except at 412 and 443 nm, where they reach 0.005 and 0.002, respectively, and root-mean-squared difference decreases from 0.006 at 412 nm to less than 0.001 at 620 and 665 nm. A first application to MODIS imagery is presented, revealing that the POLYMER algorithm is robust when pixels are contaminated by sea ice.

Classification of Mobile Laser Scanning Point Clouds from Height Features

NASA Astrophysics Data System (ADS)

Zheng, M.; Lemmens, M.; van Oosterom, P.

2017-09-01

The demand for 3D maps of cities and road networks is steadily growing and mobile laser scanning (MLS) systems are often the preferred geo-data acquisition method for capturing such scenes. Because MLS systems are mounted on cars or vans they can acquire billions of points of road scenes within a few hours of survey. Manual processing of point clouds is labour intensive and thus time consuming and expensive. Hence, the need for rapid and automated methods for 3D mapping of dense point clouds is growing exponentially. The last five years the research on automated 3D mapping of MLS data has tremendously intensified. In this paper, we present our work on automated classification of MLS point clouds. In the present stage of the research we exploited three features - two height components and one reflectance value, and achieved an overall accuracy of 73 %, which is really encouraging for further refining our approach.

Accuracy of Geophysical Parameters Derived from AIRS/AMSU as a Function of Fractional Cloud Cover

NASA Technical Reports Server (NTRS)

Susskind, Joel; Barnet, Chris; Blaisdell, John; Iredell, Lena; Keita, Fricky; Kouvaris, Lou; Molnar, Gyula; Chahine, Moustafa

2006-01-01

AIRS was launched on EOS Aqua on May 4,2002, together with AMSU A and HSB, to form a next generation polar orbiting infrared and microwave atmospheric sounding system. The primary products of AIRS/AMSU are twice daily global fields of atmospheric temperature-humidity profiles, ozone profiles, sea/land surface skin temperature, and cloud related parameters including OLR. The sounding goals of AIRS are to produce 1 km tropospheric layer mean temperatures with an rms error of lK, and layer precipitable water with an rms error of 20 percent, in cases with up to 80 percent effective cloud cover. The basic theory used to analyze Atmospheric InfraRed Sounder/Advanced Microwave Sounding Unit/Humidity Sounder Brazil (AIRS/AMSU/HSB) data in the presence of clouds, called the at-launch algorithm, was described previously. Pre-launch simulation studies using this algorithm indicated that these results should be achievable. Some modifications have been made to the at-launch retrieval algorithm as described in this paper. Sample fields of parameters retrieved from AIRS/AMSU/HSB data are presented and validated as a function of retrieved fractional cloud cover. As in simulation, the degradation of retrieval accuracy with increasing cloud cover is small and the RMS accuracy of lower tropospheric temperature retrieved with 80 percent cloud cover is about 0.5 K poorer than for clear cases. HSB failed in February 2003, and consequently HSB channel radiances are not used in the results shown in this paper. The AIRS/AMSU retrieval algorithm described in this paper, called Version 4, become operational at the Goddard DAAC (Distributed Active Archive Center) in April 2003 and is being used to analyze near-real time AIRS/AMSU data. Historical AIRS/AMSU data, going backwards from March 2005 through September 2002, is also being analyzed by the DAAC using the Version 4 algorithm.

Weakly supervised classification in high energy physics

DOE PAGES

Dery, Lucio Mwinmaarong; Nachman, Benjamin; Rubbo, Francesco; ...

2017-05-01

As machine learning algorithms become increasingly sophisticated to exploit subtle features of the data, they often become more dependent on simulations. Here, this paper presents a new approach called weakly supervised classification in which class proportions are the only input into the machine learning algorithm. Using one of the most challenging binary classification tasks in high energy physics $-$ quark versus gluon tagging $-$ we show that weakly supervised classification can match the performance of fully supervised algorithms. Furthermore, by design, the new algorithm is insensitive to any mis-modeling of discriminating features in the data by the simulation. Weakly supervisedmore » classification is a general procedure that can be applied to a wide variety of learning problems to boost performance and robustness when detailed simulations are not reliable or not available.« less

Weakly supervised classification in high energy physics

DOE Office of Scientific and Technical Information (OSTI.GOV)

Dery, Lucio Mwinmaarong; Nachman, Benjamin; Rubbo, Francesco

As machine learning algorithms become increasingly sophisticated to exploit subtle features of the data, they often become more dependent on simulations. Here, this paper presents a new approach called weakly supervised classification in which class proportions are the only input into the machine learning algorithm. Using one of the most challenging binary classification tasks in high energy physics $-$ quark versus gluon tagging $-$ we show that weakly supervised classification can match the performance of fully supervised algorithms. Furthermore, by design, the new algorithm is insensitive to any mis-modeling of discriminating features in the data by the simulation. Weakly supervisedmore » classification is a general procedure that can be applied to a wide variety of learning problems to boost performance and robustness when detailed simulations are not reliable or not available.« less

An imperialist competitive algorithm for virtual machine placement in cloud computing

NASA Astrophysics Data System (ADS)

Jamali, Shahram; Malektaji, Sepideh; Analoui, Morteza

2017-05-01

Cloud computing, the recently emerged revolution in IT industry, is empowered by virtualisation technology. In this paradigm, the user's applications run over some virtual machines (VMs). The process of selecting proper physical machines to host these virtual machines is called virtual machine placement. It plays an important role on resource utilisation and power efficiency of cloud computing environment. In this paper, we propose an imperialist competitive-based algorithm for the virtual machine placement problem called ICA-VMPLC. The base optimisation algorithm is chosen to be ICA because of its ease in neighbourhood movement, good convergence rate and suitable terminology. The proposed algorithm investigates search space in a unique manner to efficiently obtain optimal placement solution that simultaneously minimises power consumption and total resource wastage. Its final solution performance is compared with several existing methods such as grouping genetic and ant colony-based algorithms as well as bin packing heuristic. The simulation results show that the proposed method is superior to other tested algorithms in terms of power consumption, resource wastage, CPU usage efficiency and memory usage efficiency.

Foliage penetration by using 4-D point cloud data

NASA Astrophysics Data System (ADS)

Méndez Rodríguez, Javier; Sánchez-Reyes, Pedro J.; Cruz-Rivera, Sol M.

2012-06-01

Real-time awareness and rapid target detection are critical for the success of military missions. New technologies capable of detecting targets concealed in forest areas are needed in order to track and identify possible threats. Currently, LAser Detection And Ranging (LADAR) systems are capable of detecting obscured targets; however, tracking capabilities are severely limited. Now, a new LADAR-derived technology is under development to generate 4-D datasets (3-D video in a point cloud format). As such, there is a new need for algorithms that are able to process data in real time. We propose an algorithm capable of removing vegetation and other objects that may obfuscate concealed targets in a real 3-D environment. The algorithm is based on wavelets and can be used as a pre-processing step in a target recognition algorithm. Applications of the algorithm in a real-time 3-D system could help make pilots aware of high risk hidden targets such as tanks and weapons, among others. We will be using a 4-D simulated point cloud data to demonstrate the capabilities of our algorithm.

Point cloud modeling using the homogeneous transformation for non-cooperative pose estimation

NASA Astrophysics Data System (ADS)

Lim, Tae W.

2015-06-01

A modeling process to simulate point cloud range data that a lidar (light detection and ranging) sensor produces is presented in this paper in order to support the development of non-cooperative pose (relative attitude and position) estimation approaches which will help improve proximity operation capabilities between two adjacent vehicles. The algorithms in the modeling process were based on the homogeneous transformation, which has been employed extensively in robotics and computer graphics, as well as in recently developed pose estimation algorithms. Using a flash lidar in a laboratory testing environment, point cloud data of a test article was simulated and compared against the measured point cloud data. The simulated and measured data sets match closely, validating the modeling process. The modeling capability enables close examination of the characteristics of point cloud images of an object as it undergoes various translational and rotational motions. Relevant characteristics that will be crucial in non-cooperative pose estimation were identified such as shift, shadowing, perspective projection, jagged edges, and differential point cloud density. These characteristics will have to be considered in developing effective non-cooperative pose estimation algorithms. The modeling capability will allow extensive non-cooperative pose estimation performance simulations prior to field testing, saving development cost and providing performance metrics of the pose estimation concepts and algorithms under evaluation. The modeling process also provides "truth" pose of the test objects with respect to the sensor frame so that the pose estimation error can be quantified.

Knowledge-Based Object Detection in Laser Scanning Point Clouds

NASA Astrophysics Data System (ADS)

Boochs, F.; Karmacharya, A.; Marbs, A.

2012-07-01

Object identification and object processing in 3D point clouds have always posed challenges in terms of effectiveness and efficiency. In practice, this process is highly dependent on human interpretation of the scene represented by the point cloud data, as well as the set of modeling tools available for use. Such modeling algorithms are data-driven and concentrate on specific features of the objects, being accessible to numerical models. We present an approach that brings the human expert knowledge about the scene, the objects inside, and their representation by the data and the behavior of algorithms to the machine. This "understanding" enables the machine to assist human interpretation of the scene inside the point cloud. Furthermore, it allows the machine to understand possibilities and limitations of algorithms and to take this into account within the processing chain. This not only assists the researchers in defining optimal processing steps, but also provides suggestions when certain changes or new details emerge from the point cloud. Our approach benefits from the advancement in knowledge technologies within the Semantic Web framework. This advancement has provided a strong base for applications based on knowledge management. In the article we will present and describe the knowledge technologies used for our approach such as Web Ontology Language (OWL), used for formulating the knowledge base and the Semantic Web Rule Language (SWRL) with 3D processing and topologic built-ins, aiming to combine geometrical analysis of 3D point clouds, and specialists' knowledge of the scene and algorithmic processing.

Characterization of the cloud conditions at Ny-Ålesund using sensor synergy and representativeness of the observed clouds across Arctic sites

NASA Astrophysics Data System (ADS)

Nomokonova, Tatiana; Ebell, Kerstin; Löhnert, Ulrich; Maturilli, Marion

2017-04-01

Clouds are one of the crucial components of the hydrological and energy cycles and thus affecting the global climate. Their special importance in Arctic regions is defined by cloud's influence on the radiation budget. Arctic clouds usually occur at low altitudes and often contain highly concentrated tiny liquid drops. During winter, spring, and autumn periods such clouds tend to conserve the long-wave radiation in the atmosphere and, thus, produce warming of the Arctic climate. In summer though clouds efficiently scatter the solar radiation back to space and, therefore, induce a cooling effect. An accurate characterization of the net effect of clouds on the Arctic climate requires long-term and precise observations. However, only a few measurement sites exist which perform continuous, vertically resolved observations of clouds in the Arctic, e.g. in Alaska, Canada, and Greenland. These sites typically make use of a combination of different ground-based remote sensing instruments, e.g. cloud radar, ceilometer and microwave radiometer in order to characterize clouds. Within the Transregional Collaborative Research Center (TR 172) "Arctic Amplification: Climate Relevant Atmospheric and Surface Processes, and Feedback Mechanisms (AC)3" comprehensive observations of the atmospheric column are performed at the German-French Research Station AWIPEV at Ny-Ålesund, Svalbard. Ny-Ålesund is located in the warmest part of the Arctic where climate is significantly influenced by adiabatic heating from the warm ocean. Thus, measurements at Ny-Ålesund will complement our understanding of cloud formation and development in the Arctic. This particular study is devoted to the characterization of the cloud macro- and microphysical properties at Ny-Ålesund and of the atmospheric conditions, under which these clouds form and develop. To this end, the information of the various instrumentation at the AWIPEV observatory is synergistically analysed: information about the thermodynamic structure of the atmosphere is obtained from long-term radiosonde launches. In addition, continuous vertical profiles of temperature and humidity are provided by the microwave radiometer HATPRO. A set of active remote sensing instruments performs cloud observations at Ny-Ålesund: a ceilometer and a Doppler lidar operating since 2011 and 2013, respectively, are now complemented with a novel 94 GHz FMCW cloud radar. As a first step, the CLOUDNET algorithms, including a target categorization and classification, are applied to the observations. In this study, we will present a first analysis of cloud properties at Ny-Ålesund including for example cloud occurrence, cloud geometry (cloud base, cloud top, and thickness) and cloud type (liquid, ice, mixed-phase). The different types of clouds are set into context to the environmental conditions such as temperature, amount of water vapour, and liquid water. We also expect that the cloud properties strongly depend on the wind direction. The first results of this analysis will be also shown.

Discrimination of Biomass Burning Smoke and Clouds in MAIAC Algorithm

NASA Technical Reports Server (NTRS)

Lyapustin, A.; Korkin, S.; Wang, Y.; Quayle, B.; Laszlo, I.

2012-01-01

The multi-angle implementation of atmospheric correction (MAIAC) algorithm makes aerosol retrievals from MODIS data at 1 km resolution providing information about the fine scale aerosol variability. This information is required in different applications such as urban air quality analysis, aerosol source identification etc. The quality of high resolution aerosol data is directly linked to the quality of cloud mask, in particular detection of small (sub-pixel) and low clouds. This work continues research in this direction, describing a technique to detect small clouds and introducing the smoke test to discriminate the biomass burning smoke from the clouds. The smoke test relies on a relative increase of aerosol absorption at MODIS wavelength 0.412 micrometers as compared to 0.47-0.67 micrometers due to multiple scattering and enhanced absorption by organic carbon released during combustion. This general principle has been successfully used in the OMI detection of absorbing aerosols based on UV measurements. This paper provides the algorithm detail and illustrates its performance on two examples of wildfires in US Pacific North-West and in Georgia/Florida of 2007.

Improvements in Cloud Remote Sensing from Fusing VIIRS and CrIS data

NASA Astrophysics Data System (ADS)

Heidinger, A. K.; Walther, A.; Lindsey, D. T.; Li, Y.; NOH, Y. J.; Botambekov, D.; Miller, S. D.; Foster, M. J.

2016-12-01

In the fall of 2016, NOAA began the operational production of cloud products from the S-NPP Visible and Infrared Imaging Radiometer Suite (VIIRS) using the NOAA Enterprise Algorithms. VIIRS, while providing unprecedented spatial resolution and imaging clarity, does lack certain IR channels that are beneficial to cloud remote sensing. At the UW Space Science and Engineering Center (SSEC), tools were written to generate the missing IR channels from the Cross Track Infrared Sounder (CrIS) and to map them into the VIIRS swath. The NOAA Enterprise Algorithms are also implemented into the NESDIS CLAVR-x system. CLAVR-x has been modified to use the fused VIIRS and CrIS data. This presentation will highlight the benefits offered by the CrIS data to the NOAA Enterprise Algorithms. In addition, these benefits also have enabled the generation of 3D cloud retrievals to support the request from the National Weather Service (NWS) for a Cloud Cover Layers product. Lastly, the benefits of using VIIRS and CrIS for achieving consistency with GOES-R will also be demonstrated.

Assessing and Correcting Topographic Effects on Forest Canopy Height Retrieval Using Airborne LiDAR Data

PubMed Central

Duan, Zhugeng; Zhao, Dan; Zeng, Yuan; Zhao, Yujin; Wu, Bingfang; Zhu, Jianjun

2015-01-01

Topography affects forest canopy height retrieval based on airborne Light Detection and Ranging (LiDAR) data a lot. This paper proposes a method for correcting deviations caused by topography based on individual tree crown segmentation. The point cloud of an individual tree was extracted according to crown boundaries of isolated individual trees from digital orthophoto maps (DOMs). Normalized canopy height was calculated by subtracting the elevation of centres of gravity from the elevation of point cloud. First, individual tree crown boundaries are obtained by carrying out segmentation on the DOM. Second, point clouds of the individual trees are extracted based on the boundaries. Third, precise DEM is derived from the point cloud which is classified by a multi-scale curvature classification algorithm. Finally, a height weighted correction method is applied to correct the topological effects. The method is applied to LiDAR data acquired in South China, and its effectiveness is tested using 41 field survey plots. The results show that the terrain impacts the canopy height of individual trees in that the downslope side of the tree trunk is elevated and the upslope side is depressed. This further affects the extraction of the location and crown of individual trees. A strong correlation was detected between the slope gradient and the proportions of returns with height differences more than 0.3, 0.5 and 0.8 m in the total returns, with coefficient of determination R2 of 0.83, 0.76, and 0.60 (n = 41), respectively. PMID:26016907

Pattern recognition analysis of polar clouds during summer and winter

NASA Technical Reports Server (NTRS)

Ebert, Elizabeth E.

1992-01-01

A pattern recognition algorithm is demonstrated which classifies eighteen surface and cloud types in high-latitude AVHRR imagery based on several spectral and textural features, then estimates the cloud properties (fractional coverage, albedo, and brightness temperature) using a hybrid histogram and spatial coherence technique. The summertime version of the algorithm uses both visible and infrared data (AVHRR channels 1-4), while the wintertime version uses only infrared data (AVHRR channels 3-5). Three days of low-resolution AVHRR imagery from the Arctic and Antarctic during January and July 1984 were analyzed for cloud type and fractional coverage. The analysis showed significant amounts of high cloudiness in the Arctic during one day in winter. The Antarctic summer scene was characterized by heavy cloud cover in the southern ocean and relatively clear conditions in the continental interior. A large region of extremely low brightness temperatures in East Antarctica during winter suggests the presence of polar stratospheric cloud.

Applying Cost-Sensitive Extreme Learning Machine and Dissimilarity Integration to Gene Expression Data Classification.

PubMed

Liu, Yanqiu; Lu, Huijuan; Yan, Ke; Xia, Haixia; An, Chunlin

2016-01-01

Embedding cost-sensitive factors into the classifiers increases the classification stability and reduces the classification costs for classifying high-scale, redundant, and imbalanced datasets, such as the gene expression data. In this study, we extend our previous work, that is, Dissimilar ELM (D-ELM), by introducing misclassification costs into the classifier. We name the proposed algorithm as the cost-sensitive D-ELM (CS-D-ELM). Furthermore, we embed rejection cost into the CS-D-ELM to increase the classification stability of the proposed algorithm. Experimental results show that the rejection cost embedded CS-D-ELM algorithm effectively reduces the average and overall cost of the classification process, while the classification accuracy still remains competitive. The proposed method can be extended to classification problems of other redundant and imbalanced data.

Classification of Urban Feature from Unmanned Aerial Vehicle Images Using Gasvm Integration and Multi-Scale Segmentation

NASA Astrophysics Data System (ADS)

Modiri, M.; Salehabadi, A.; Mohebbi, M.; Hashemi, A. M.; Masumi, M.

2015-12-01

The use of UAV in the application of photogrammetry to obtain cover images and achieve the main objectives of the photogrammetric mapping has been a boom in the region. The images taken from REGGIOLO region in the province of, Italy Reggio -Emilia by UAV with non-metric camera Canon Ixus and with an average height of 139.42 meters were used to classify urban feature. Using the software provided SURE and cover images of the study area, to produce dense point cloud, DSM and Artvqvtv spatial resolution of 10 cm was prepared. DTM area using Adaptive TIN filtering algorithm was developed. NDSM area was prepared with using the difference between DSM and DTM and a separate features in the image stack. In order to extract features, using simultaneous occurrence matrix features mean, variance, homogeneity, contrast, dissimilarity, entropy, second moment, and correlation for each of the RGB band image was used Orthophoto area. Classes used to classify urban problems, including buildings, trees and tall vegetation, grass and vegetation short, paved road and is impervious surfaces. Class consists of impervious surfaces such as pavement conditions, the cement, the car, the roof is stored. In order to pixel-based classification and selection of optimal features of classification was GASVM pixel basis. In order to achieve the classification results with higher accuracy and spectral composition informations, texture, and shape conceptual image featureOrthophoto area was fencing. The segmentation of multi-scale segmentation method was used.it belonged class. Search results using the proposed classification of urban feature, suggests the suitability of this method of classification complications UAV is a city using images. The overall accuracy and kappa coefficient method proposed in this study, respectively, 47/93% and 84/91% was.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Deng, Min; Kollias, Pavlos; Feng, Zhe

The motivation for this research is to develop a precipitation classification and rain rate estimation method using cloud radar-only measurements for Atmospheric Radiation Measurement (ARM) long-term cloud observation analysis, which are crucial and unique for studying cloud lifecycle and precipitation features under different weather and climate regimes. Based on simultaneous and collocated observations of the Ka-band ARM zenith radar (KAZR), two precipitation radars (NCAR S-PolKa and Texas A&M University SMART-R), and surface precipitation during the DYNAMO/AMIE field campaign, a new cloud radar-only based precipitation classification and rain rate estimation method has been developed and evaluated. The resulting precipitation classification ismore » equivalent to those collocated SMART-R and S-PolKa observations. Both cloud and precipitation radars detected about 5% precipitation occurrence during this period. The convective (stratiform) precipitation fraction is about 18% (82%). The 2-day collocated disdrometer observations show an increased number concentration of large raindrops in convective rain compared to dominant concentration of small raindrops in stratiform rain. The composite distributions of KAZR reflectivity and Doppler velocity also show two distinct structures for convective and stratiform rain. These indicate that the method produces physically consistent results for two types of rain. The cloud radar-only rainfall estimation is developed based on the gradient of accumulative radar reflectivity below 1 km, near-surface Ze, and collocated surface rainfall (R) measurement. The parameterization is compared with the Z-R exponential relation. The relative difference between estimated and surface measured rainfall rate shows that the two-parameter relation can improve rainfall estimation.« less

Evaluation of multi-layer cloud detection based on MODIS CO2-slicing algorithm with CALIPSO-CloudSat measurements.

NASA Astrophysics Data System (ADS)

Viudez-Mora, A.; Kato, S.; Smith, W. L., Jr.; Chang, F. L.

2016-12-01

Knowledge of the vertical cloud distribution is important for a variety of climate and weather applications. The cloud overlapping variations greatly influence the atmospheric heating/cooling rates, with implications for the surface-troposphere radiative balance, global circulation and precipitation. Additionally, an accurate knowledge of the multi-layer cloud distribution in real-time can be used in applications such safety condition for aviation through storms and adverse weather conditions. In this study, we evaluate a multi-layered cloud algorithm (Chang et al. 2005) based on MODIS measurements aboard Aqua satellite (MCF). This algorithm uses the CO2-slicing technique combined with cloud properties determined from VIS, IR and NIR channels to locate high thin clouds over low-level clouds, and retrieve the τ of each layer. We use CALIPSO (Winker et. al, 2010) and CloudSat (Stephens et. al, 2002) (CLCS) derived cloud vertical profiles included in the C3M data product (Kato et al. 2010) to evaluate MCF derived multi-layer cloud properties. We focus on 2 layer overlapping and 1-layer clouds identified by the active sensors and investigate how well these systems are identified by the MODIS multi-layer technique. The results show that for these multi-layered clouds identified by CLCS, the MCF correctly identifies about 83% of the cases as multi-layer. However, it is found that the upper CTH is underestimated by about 2.6±0.4 km, because the CO2-slicing technique is not as sensitive to the cloud physical top as the CLCS. The lower CTH agree better with differences found to be about 1.2±0.5 km. Another outstanding issue for the MCF approach is the large number of multi-layer false alarms that occur in single-layer conditions. References: Chang, F.-L., and Z. Li, 2005: A new method for detection of cirrus overlapping water clouds and determination of their optical properties. J. Atmos. Sci., 62. Kato, S., et al. (2010), Relationships among cloud occurrence frequency, overlap, and effective thickness derived from CALIPSO and CloudSat merged cloud vertical profiles, J. Geophys. Res., 115. Stephens, G. L., et al. (2002), The CloudSat mission and A-Train, Bull. Am. Meteorol. Soc., 83. Winker, D. M., et al., 2010: The CALIPSO Mission: A global 3D view of aerosols and clouds. Bull. Amer. Meteor. Soc., 91.

Robust Group Sparse Beamforming for Multicast Green Cloud-RAN With Imperfect CSI

NASA Astrophysics Data System (ADS)

Shi, Yuanming; Zhang, Jun; Letaief, Khaled B.

2015-09-01

In this paper, we investigate the network power minimization problem for the multicast cloud radio access network (Cloud-RAN) with imperfect channel state information (CSI). The key observation is that network power minimization can be achieved by adaptively selecting active remote radio heads (RRHs) via controlling the group-sparsity structure of the beamforming vector. However, this yields a non-convex combinatorial optimization problem, for which we propose a three-stage robust group sparse beamforming algorithm. In the first stage, a quadratic variational formulation of the weighted mixed l1/l2-norm is proposed to induce the group-sparsity structure in the aggregated beamforming vector, which indicates those RRHs that can be switched off. A perturbed alternating optimization algorithm is then proposed to solve the resultant non-convex group-sparsity inducing optimization problem by exploiting its convex substructures. In the second stage, we propose a PhaseLift technique based algorithm to solve the feasibility problem with a given active RRH set, which helps determine the active RRHs. Finally, the semidefinite relaxation (SDR) technique is adopted to determine the robust multicast beamformers. Simulation results will demonstrate the convergence of the perturbed alternating optimization algorithm, as well as, the effectiveness of the proposed algorithm to minimize the network power consumption for multicast Cloud-RAN.

Two new algorithms to combine kriging with stochastic modelling

NASA Astrophysics Data System (ADS)

Venema, Victor; Lindau, Ralf; Varnai, Tamas; Simmer, Clemens

2010-05-01

Two main groups of statistical methods used in the Earth sciences are geostatistics and stochastic modelling. Geostatistical methods, such as various kriging algorithms, aim at estimating the mean value for every point as well as possible. In case of sparse measurements, such fields have less variability at small scales and a narrower distribution as the true field. This can lead to biases if a nonlinear process is simulated driven by such a kriged field. Stochastic modelling aims at reproducing the statistical structure of the data in space and time. One of the stochastic modelling methods, the so-called surrogate data approach, replicates the value distribution and power spectrum of a certain data set. While stochastic methods reproduce the statistical properties of the data, the location of the measurement is not considered. This requires the use of so-called constrained stochastic models. Because radiative transfer through clouds is a highly nonlinear process, it is essential to model the distribution (e.g. of optical depth, extinction, liquid water content or liquid water path) accurately. In addition, the correlations within the cloud field are important, especially because of horizontal photon transport. This explains the success of surrogate cloud fields for use in 3D radiative transfer studies. Up to now, however, we could only achieve good results for the radiative properties averaged over the field, but not for a radiation measurement located at a certain position. Therefore we have developed a new algorithm that combines the accuracy of stochastic (surrogate) modelling with the positioning capabilities of kriging. In this way, we can automatically profit from the large geostatistical literature and software. This algorithm is similar to the standard iterative amplitude adjusted Fourier transform (IAAFT) algorithm, but has an additional iterative step in which the surrogate field is nudged towards the kriged field. The nudging strength is gradually reduced to zero during successive iterations. A second algorithm, which we call step-wise kriging, pursues the same aim. Each time the kriging algorithm estimates a value, noise is added to it, after which this new point is accounted for in the estimation of all the later points. In this way, the autocorrelation of the step-krigged field is close to that found in the pseudo measurements. The amount of noise is determined by the kriging uncertainty. The algorithms are tested on cloud fields from large eddy simulations (LES). On these clouds, a measurement is simulated. From these pseudo-measurements, we estimated the power spectrum for the surrogates, the semi-variogram for the (stepwise) kriging and the distribution. Furthermore, the pseudo-measurement is kriged. Because we work with LES clouds and the truth is known, we can validate the algorithm by performing 3D radiative transfer calculations on the original LES clouds and on the two new types of stochastic clouds. For comparison, also the radiative properties of the kriged fields and standard surrogate fields are computed. Preliminary results show that both algorithms reproduce the structure of the original clouds well, and the minima and maxima are located where the pseudo-measurements see them. The main problem for the quality of the structure and the root mean square error is the amount of data, which is especially very limited in case of just one zenith pointing measurement.

Cloud Properties of CERES-MODIS Edition 4 and CERES-VIIRS Edition 1

NASA Technical Reports Server (NTRS)

Sun-Mack, Sunny; Minnis, Patrick; Chang, Fu-Lung; Hong, Gang; Arduini, Robert; Chen, Yan; Trepte, Qing; Yost, Chris; Smith, Rita; Brown, Ricky;

Sensitivity of Marine Warm Cloud Retrieval Statistics to Algorithm Choices: Examples from MODIS Collection 6

NASA Astrophysics Data System (ADS)

Platnick, S.; Wind, G.; Zhang, Z.; Ackerman, S. A.; Maddux, B. C.

2012-12-01

The optical and microphysical structure of warm boundary layer marine clouds is of fundamental importance for understanding a variety of cloud radiation and precipitation processes. With the advent of MODIS (Moderate Resolution Imaging Spectroradiometer) on the NASA EOS Terra and Aqua platforms, simultaneous global/daily 1km retrievals of cloud optical thickness and effective particle size are provided, as well as the derived water path. In addition, the cloud product (MOD06/MYD06 for MODIS Terra and Aqua, respectively) provides separate effective radii results using the 1.6, 2.1, and 3.7 μm spectral channels. Cloud retrieval statistics are highly sensitive to how a pixel identified as being "not-clear" by a cloud mask (e.g., the MOD35/MYD35 product) is determined to be useful for an optical retrieval based on a 1-D cloud model. The Collection 5 MODIS retrieval algorithm removed pixels associated with cloud edges (defined by immediate adjacency to "clear" MOD/MYD35 pixels) as well as ocean pixels with partly cloudy elements in the 250m MODIS cloud mask - part of the so-called Clear Sky Restoral (CSR) algorithm. Collection 6 attempts retrievals for those two pixel populations, but allows a user to isolate or filter out the populations via CSR pixel-level Quality Assessment (QA) assignments. In this paper, using the preliminary Collection 6 MOD06 product, we present global and regional statistical results of marine warm cloud retrieval sensitivities to the cloud edge and 250m partly cloudy pixel populations. As expected, retrievals for these pixels are generally consistent with a breakdown of the 1D cloud model. While optical thickness for these suspect pixel populations may have some utility for radiative studies, the retrievals should be used with extreme caution for process and microphysical studies.

Determination of cloud liquid water content using the SSM/I

NASA Technical Reports Server (NTRS)

Alishouse, John C.; Snider, Jack B.; Westwater, Ed R.; Swift, Calvin T.; Ruf, Christopher S.

1990-01-01

As part of a calibration/validation effort for the special sensor microwave/imager (SSM/I), coincident observations of SSM/I brightness temperatures and surface-based observations of cloud liquid water were obtained. These observations were used to validate initial algorithms and to derive an improved algorithm. The initial algorithms were divided into latitudinal-, seasonal-, and surface-type zones. It was found that these initial algorithms, which were of the D-matrix type, did not yield sufficiently accurate results. The surface-based measurements of channels were investigated; however, the 85V channel was excluded because of excessive noise. It was found that there is no significant correlation between the SSM/I brightness temperatures and the surface-based cloud liquid water determination when the background surface is land or snow. A high correlation was found between brightness temperatures and ground-based measurements over the ocean.

A technique for global monitoring of net solar irradiance at the ocean surface. I - Model

NASA Technical Reports Server (NTRS)

Frouin, Robert; Chertock, Beth

1992-01-01

An accurate long-term (84-month) climatology of net surface solar irradiance over the global oceans from Nimbus-7 earth radiation budget (ERB) wide-field-of-view planetary-albedo data is generated via an algorithm based on radiative transfer theory. Net surface solar irradiance is computed as the difference between the top-of-atmosphere incident solar irradiance (known) and the sum of the solar irradiance reflected back to space by the earth-atmosphere system (observed) and the solar irradiance absorbed by atmospheric constituents (modeled). It is shown that the effects of clouds and clear-atmosphere constituents can be decoupled on a monthly time scale, which makes it possible to directly apply the algorithm with monthly averages of ERB planetary-albedo data. Compared theoretically with the algorithm of Gautier et al. (1980), the present algorithm yields higher solar irradiance values in clear and thin cloud conditions and lower values in thick cloud conditions.

Active sensor synergy for arctic cloud microphysics

NASA Astrophysics Data System (ADS)

Sato, Kaori; Okamoto, Hajime; Katagiri, Shuichiro; Shiobara, Masataka; Yabuki, Masanori; Takano, Toshiaki

2018-04-01

In this study, we focus on the retrieval of liquid and ice-phase cloud microphysics from spaceborne and ground-based lidar-cloud radar synergy. As an application of the cloud retrieval algorithm developed for the EarthCARE satellite mission (JAXA-ESA) [1], the derived statistics of cloud microphysical properties in high latitudes and their relation to the Arctic climate are investigated.

Investigating the differences of cirrus cloud properties in nucleation, growth and sublimation regions based on airborne water vapor lidar measurements

NASA Astrophysics Data System (ADS)

Urbanek, Benedikt; Groß, Silke; Wirth, Martin

2017-04-01

Cirrus clouds impose high uncertainties on weather and climate prediction, as knowledge on important processes is still incomplete. For instance it remains unclear how cloud optical, microphysical, and radiative properties change as the cirrus evolves. To gain better understanding of cirrus clouds, their optical and microphysical properties and their changes with cirrus cloud evolution the ML-CIRRUS campaign was conducted in March and April 2014. Measurements with a combined in-situ and remote sensing payload were performed with the German research aircraft HALO based in Oberpfaffenhofen. 16 research flights with altogether 88 flight hours were performed over the North-Atlantic, western and central Europe to probe different cirrus cloud regimes and cirrus clouds at different stages of evolution. One of the key remotes sensing instruments during ML-CIRRUS was the airborne differential absorption and high spectral lidar system WALES. It measures the 2-dimensional distribution of water vapor inside and outside of cirrus clouds as well as the optical properties of the clouds. Bases on these airborne lidar measurements a novel classification scheme to derive the stage of cirrus cloud evolution was developed. It identifies regions of ice nucleation, particle growth by deposition of water vapor, and ice sublimation. This method is used to investigate differences in the distribution and value of optical properties as well as in the distribution of water vapor and relative humidity depending on the stage of evolution of the cloud. We will present the lidar based classification scheme and its application on a wave driven cirrus cloud case, and we will show first results of the dependence of optical cloud properties and relative humidity distributions on the determined stage of evolution.

Reduction from cost-sensitive ordinal ranking to weighted binary classification.

PubMed

Lin, Hsuan-Tien; Li, Ling

2012-05-01

We present a reduction framework from ordinal ranking to binary classification. The framework consists of three steps: extracting extended examples from the original examples, learning a binary classifier on the extended examples with any binary classification algorithm, and constructing a ranker from the binary classifier. Based on the framework, we show that a weighted 0/1 loss of the binary classifier upper-bounds the mislabeling cost of the ranker, both error-wise and regret-wise. Our framework allows not only the design of good ordinal ranking algorithms based on well-tuned binary classification approaches, but also the derivation of new generalization bounds for ordinal ranking from known bounds for binary classification. In addition, our framework unifies many existing ordinal ranking algorithms, such as perceptron ranking and support vector ordinal regression. When compared empirically on benchmark data sets, some of our newly designed algorithms enjoy advantages in terms of both training speed and generalization performance over existing algorithms. In addition, the newly designed algorithms lead to better cost-sensitive ordinal ranking performance, as well as improved listwise ranking performance.

Classification of Medical Datasets Using SVMs with Hybrid Evolutionary Algorithms Based on Endocrine-Based Particle Swarm Optimization and Artificial Bee Colony Algorithms.

PubMed

Lin, Kuan-Cheng; Hsieh, Yi-Hsiu

2015-10-01

The classification and analysis of data is an important issue in today's research. Selecting a suitable set of features makes it possible to classify an enormous quantity of data quickly and efficiently. Feature selection is generally viewed as a problem of feature subset selection, such as combination optimization problems. Evolutionary algorithms using random search methods have proven highly effective in obtaining solutions to problems of optimization in a diversity of applications. In this study, we developed a hybrid evolutionary algorithm based on endocrine-based particle swarm optimization (EPSO) and artificial bee colony (ABC) algorithms in conjunction with a support vector machine (SVM) for the selection of optimal feature subsets for the classification of datasets. The results of experiments using specific UCI medical datasets demonstrate that the accuracy of the proposed hybrid evolutionary algorithm is superior to that of basic PSO, EPSO and ABC algorithms, with regard to classification accuracy using subsets with a reduced number of features.

Accuracy Assessments of Cloud Droplet Size Retrievals from Polarized Reflectance Measurements by the Research Scanning Polarimeter

NASA Technical Reports Server (NTRS)

Alexandrov, Mikhail Dmitrievic; Cairns, Brian; Emde, Claudia; Ackerman, Andrew S.; vanDiedenhove, Bastiaan

2012-01-01

We present an algorithm for the retrieval of cloud droplet size distribution parameters (effective radius and variance) from the Research Scanning Polarimeter (RSP) measurements. The RSP is an airborne prototype for the Aerosol Polarimetery Sensor (APS), which was on-board of the NASA Glory satellite. This instrument measures both polarized and total reflectance in 9 spectral channels with central wavelengths ranging from 410 to 2260 nm. The cloud droplet size retrievals use the polarized reflectance in the scattering angle range between 135deg and 165deg, where they exhibit the sharply defined structure known as the rain- or cloud-bow. The shape of the rainbow is determined mainly by the single scattering properties of cloud particles. This significantly simplifies both forward modeling and inversions, while also substantially reducing uncertainties caused by the aerosol loading and possible presence of undetected clouds nearby. In this study we present the accuracy evaluation of our algorithm based on the results of sensitivity tests performed using realistic simulated cloud radiation fields.

Cloudy Sounding and Cloud-Top Height Retrieval From AIRS Alone Single Field-of-View Radiance Measurements

NASA Technical Reports Server (NTRS)

Weisz, Elisabeth; Li, Jun; Li, Jinlong; Zhou, Daniel K.; Huang, Hung-Lung; Goldberg, Mitchell D.; Yang, Ping

2007-01-01

High-spectral resolution measurements from the Atmospheric Infrared Sounder (AIRS) onboard the EOS (Earth Observing System) Aqua satellite provide unique information about atmospheric state, surface and cloud properties. This paper presents an AIRS alone single field-of-view (SFOV) retrieval algorithm to simultaneously retrieve temperature, humidity and ozone profiles under all weather conditions, as well as cloud top pressure (CTP) and cloud optical thickness (COT) under cloudy skies. For optically thick cloud conditions the above-cloud soundings are derived, whereas for clear skies and optically thin cloud conditions the profiles are retrieved from 0.005 hPa down to the earth's surface. Initial validation has been conducted by using the operational MODIS (Moderate Resolution Imaging Spectroradiometer) product, ECMWF (European Center of Medium range Weather Forecasts) analysis fields and radiosonde observations (RAOBs). These inter-comparisons clearly demonstrate the potential of this algorithm to process data from 38 high-spectral infrared (IR) sounder instruments.

Workflow as a Service in the Cloud: Architecture and Scheduling Algorithms

PubMed Central

Wang, Jianwu; Korambath, Prakashan; Altintas, Ilkay; Davis, Jim; Crawl, Daniel

2017-01-01

With more and more workflow systems adopting cloud as their execution environment, it becomes increasingly challenging on how to efficiently manage various workflows, virtual machines (VMs) and workflow execution on VM instances. To make the system scalable and easy-to-extend, we design a Workflow as a Service (WFaaS) architecture with independent services. A core part of the architecture is how to efficiently respond continuous workflow requests from users and schedule their executions in the cloud. Based on different targets, we propose four heuristic workflow scheduling algorithms for the WFaaS architecture, and analyze the differences and best usages of the algorithms in terms of performance, cost and the price/performance ratio via experimental studies. PMID:29399237

Simple-random-sampling-based multiclass text classification algorithm.

PubMed

Liu, Wuying; Wang, Lin; Yi, Mianzhu

2014-01-01

Multiclass text classification (MTC) is a challenging issue and the corresponding MTC algorithms can be used in many applications. The space-time overhead of the algorithms must be concerned about the era of big data. Through the investigation of the token frequency distribution in a Chinese web document collection, this paper reexamines the power law and proposes a simple-random-sampling-based MTC (SRSMTC) algorithm. Supported by a token level memory to store labeled documents, the SRSMTC algorithm uses a text retrieval approach to solve text classification problems. The experimental results on the TanCorp data set show that SRSMTC algorithm can achieve the state-of-the-art performance at greatly reduced space-time requirements.

Automated cloud classification using a ground based infra-red camera and texture analysis techniques

NASA Astrophysics Data System (ADS)

Rumi, Emal; Kerr, David; Coupland, Jeremy M.; Sandford, Andrew P.; Brettle, Mike J.

2013-10-01

Clouds play an important role in influencing the dynamics of local and global weather and climate conditions. Continuous monitoring of clouds is vital for weather forecasting and for air-traffic control. Convective clouds such as Towering Cumulus (TCU) and Cumulonimbus clouds (CB) are associated with thunderstorms, turbulence and atmospheric instability. Human observers periodically report the presence of CB and TCU clouds during operational hours at airports and observatories; however such observations are expensive and time limited. Robust, automatic classification of cloud type using infrared ground-based instrumentation offers the advantage of continuous, real-time (24/7) data capture and the representation of cloud structure in the form of a thermal map, which can greatly help to characterise certain cloud formations. The work presented here utilised a ground based infrared (8-14 μm) imaging device mounted on a pan/tilt unit for capturing high spatial resolution sky images. These images were processed to extract 45 separate textural features using statistical and spatial frequency based analytical techniques. These features were used to train a weighted k-nearest neighbour (KNN) classifier in order to determine cloud type. Ground truth data were obtained by inspection of images captured simultaneously from a visible wavelength colour camera at the same installation, with approximately the same field of view as the infrared device. These images were classified by a trained cloud observer. Results from the KNN classifier gave an encouraging success rate. A Probability of Detection (POD) of up to 90% with a Probability of False Alarm (POFA) as low as 16% was achieved.

New Developments in the SCIAMACHY L2 Ground Processor

NASA Astrophysics Data System (ADS)

Gretschany, Sergei; Lichtenberg, Günter; Meringer, Markus; Theys, Nicolas; Lerot, Christophe; Liebing, Patricia; Noel, Stefan; Dehn, Angelika; Fehr, Thorsten

2016-04-01

SCIAMACHY (SCanning Imaging Absorption spectroMeter for Atmospheric ChartographY) aboard ESA's environmental satellite ENVISAT observed the Earth's atmosphere in limb, nadir, and solar/lunar occultation geometries covering the UV-Visible to NIR spectral range. It is a joint project of Germany, the Netherlands and Belgium and was launched in February 2002. SCIAMACHY doubled its originally planned in-orbit lifetime of five years before the communication to ENVISAT was severed in April 2012, and the mission entered its post-operational phase. In order to preserve the best quality of the outstanding data recorded by SCIAMACHY, data processors are still being updated. This presentation will highlight three new developments that are currently being incorporated into the forthcoming Version 7 of ESA's operational Level 2 processor: 1. Tropospheric BrO, a new retrieval based on the scientific algorithm of (Theys et al., 2011). This algorithm had been originally developed for the GOME-2 sensor and later adapted for SCIAMACHY. The main principle of the new algorithm is to utilize BrO total columns (already an operational product) and split them into stratospheric VCDstrat and tropospheric VCDtrop fractions. BrO VCDstrat is determined from a climatological approach, driven by SCIAMACHY O3 and NO2 observations. VCDtrop is then determined simply as a difference: VCDtrop = VCDtotal - VCDstrat. 2. Improved cloud flagging using limb measurements (Liebing, 2015). Limb cloud flags are already part of the SCIAMACHY L2 product. They are currently calculated employing the scientific algorithm developed by (Eichmann et al., 2015). Clouds are categorized into four types: water, ice, polar stratospheric and noctilucent clouds. High atmospheric aerosol loadings, however, often lead to spurious cloud flags, when aerosols had been misidentified as clouds. The new algorithm will better discriminate between aerosol and clouds. It will also have a higher sensitivity w.r.t. thin clouds. 3. A new, future-proof file format for the level 2 product based on NetCDF. Although the final concept for the new format is still under discussion within the SCIAMACHY Quality Working Group, main features of the new format have already been clarified. The data format should be aligned and harmonized with other missions (esp. Sentinels and GOME-1). Splitting of the L2 products into profile and column products is also considered. Additionally, reading routines for the new formats will be developed and provided. References: K.-U. Eichmann et al., Global cloud top height retrieval using SCIAMACHY limb spectra: model studies and first results, Atmos. Meas. Tech. Discuss., 8, 8295-8352, 2015. P. Liebing, New Limb Cloud Detection Algorithm Theoretical Basis Document, 2015. N. Theys et al., Global observations of tropospheric BrO columns using GOME-2 satellite data, Atmos. Chem. Phys., 11, 1791-1811, 2011.

Big Data: A Parallel Particle Swarm Optimization-Back-Propagation Neural Network Algorithm Based on MapReduce.

PubMed

Cao, Jianfang; Cui, Hongyan; Shi, Hao; Jiao, Lijuan

2016-01-01

A back-propagation (BP) neural network can solve complicated random nonlinear mapping problems; therefore, it can be applied to a wide range of problems. However, as the sample size increases, the time required to train BP neural networks becomes lengthy. Moreover, the classification accuracy decreases as well. To improve the classification accuracy and runtime efficiency of the BP neural network algorithm, we proposed a parallel design and realization method for a particle swarm optimization (PSO)-optimized BP neural network based on MapReduce on the Hadoop platform using both the PSO algorithm and a parallel design. The PSO algorithm was used to optimize the BP neural network's initial weights and thresholds and improve the accuracy of the classification algorithm. The MapReduce parallel programming model was utilized to achieve parallel processing of the BP algorithm, thereby solving the problems of hardware and communication overhead when the BP neural network addresses big data. Datasets on 5 different scales were constructed using the scene image library from the SUN Database. The classification accuracy of the parallel PSO-BP neural network algorithm is approximately 92%, and the system efficiency is approximately 0.85, which presents obvious advantages when processing big data. The algorithm proposed in this study demonstrated both higher classification accuracy and improved time efficiency, which represents a significant improvement obtained from applying parallel processing to an intelligent algorithm on big data.

Optimized extreme learning machine for urban land cover classification using hyperspectral imagery

NASA Astrophysics Data System (ADS)

Su, Hongjun; Tian, Shufang; Cai, Yue; Sheng, Yehua; Chen, Chen; Najafian, Maryam

2017-12-01

This work presents a new urban land cover classification framework using the firefly algorithm (FA) optimized extreme learning machine (ELM). FA is adopted to optimize the regularization coefficient C and Gaussian kernel σ for kernel ELM. Additionally, effectiveness of spectral features derived from an FA-based band selection algorithm is studied for the proposed classification task. Three sets of hyperspectral databases were recorded using different sensors, namely HYDICE, HyMap, and AVIRIS. Our study shows that the proposed method outperforms traditional classification algorithms such as SVM and reduces computational cost significantly.

Auto-SEIA: simultaneous optimization of image processing and machine learning algorithms

NASA Astrophysics Data System (ADS)

Negro Maggio, Valentina; Iocchi, Luca

2015-02-01

Object classification from images is an important task for machine vision and it is a crucial ingredient for many computer vision applications, ranging from security and surveillance to marketing. Image based object classification techniques properly integrate image processing and machine learning (i.e., classification) procedures. In this paper we present a system for automatic simultaneous optimization of algorithms and parameters for object classification from images. More specifically, the proposed system is able to process a dataset of labelled images and to return a best configuration of image processing and classification algorithms and of their parameters with respect to the accuracy of classification. Experiments with real public datasets are used to demonstrate the effectiveness of the developed system.

An Online 3D Database System for Endangered Architectural and Archaeological Heritage in the South-Eastern Mediterranean

NASA Astrophysics Data System (ADS)

Abate, D.; Avgousti, A.; Faka, M.; Hermon, S.; Bakirtzis, N.; Christofi, P.

2017-10-01

This study compares performance of aerial image based point clouds (IPCs) and light detection and ranging (LiDAR) based point clouds in detection of thinnings and clear cuts in forests. IPCs are an appealing method to update forest resource data, because of their accuracy in forest height estimation and cost-efficiency of aerial image acquisition. We predicted forest changes over a period of three years by creating difference layers that displayed the difference in height or volume between the initial and subsequent time points. Both IPCs and LiDAR data were used in this process. The IPCs were constructed with the Semi-Global Matching (SGM) algorithm. Difference layers were constructed by calculating differences in fitted height or volume models or in canopy height models (CHMs) from both time points. The LiDAR-derived digital terrain model (DTM) was used to scale heights to above ground level. The study area was classified in logistic regression into the categories ClearCut, Thinning or NoChange with the values from the difference layers. We compared the predicted changes with the true changes verified in the field, and obtained at best a classification accuracy for clear cuts 93.1 % with IPCs and 91.7 % with LiDAR data. However, a classification accuracy for thinnings was only 8.0 % with IPCs. With LiDAR data 41.4 % of thinnings were detected. In conclusion, the LiDAR data proved to be more accurate method to predict the minor changes in forests than IPCs, but both methods are useful in detection of major changes.

Full-polarization radar remote sensing and data mining for tropical crops mapping: a successful SVM-based classification model

NASA Astrophysics Data System (ADS)

Denize, J.; Corgne, S.; Todoroff, P.; LE Mezo, L.

2015-12-01

In Reunion, a tropical island of 2,512 km², 700 km east of Madagascar in the Indian Ocean, constrained by a rugged relief, agricultural sectors are competing in highly fragmented agricultural land constituted by heterogeneous farming systems from corporate to small-scale farming. Policymakers, planners and institutions are in dire need of reliable and updated land use references. Actually conventional land use mapping methods are inefficient under the tropic with frequent cloud cover and loosely synchronous vegetative cycles of the crops due to a constant temperature. This study aims to provide an appropriate method for the identification and mapping of tropical crops by remote sensing. For this purpose, we assess the potential of polarimetric SAR imagery associated with associated with machine learning algorithms. The method has been developed and tested on a study area of 25*25 km thanks to 6 RADARSAT-2 images in 2014 in full-polarization. A set of radar indicators (backscatter coefficient, bands ratios, indices, polarimetric decompositions (Freeman-Durden, Van zyl, Yamaguchi, Cloude and Pottier, Krogager), texture, etc.) was calculated from the coherency matrix. A random forest procedure allowed the selection of the most important variables on each images to reduce the dimension of the dataset and the processing time. Support Vector Machines (SVM), allowed the classification of these indicators based on a learning database created from field observations in 2013. The method shows an overall accuracy of 88% with a Kappa index of 0.82 for the identification of four major crops.

Quantifying Uncertainties in Mass-Dimensional Relationships Through a Comparison Between CloudSat and SPartICus Reflectivity Factors

NASA Astrophysics Data System (ADS)

Mascio, J.; Mace, G. G.

2015-12-01

CloudSat and CALIPSO, two of the satellites in the A-Train constellation, use algorithms to calculate the scattering properties of small cloud particles, such as the T-matrix method. Ice clouds (i.e. cirrus) cause problems with these cloud property retrieval algorithms because of their variability in ice mass as a function of particle size. Assumptions regarding the microphysical properties, such as mass-dimensional (m-D) relationships, are often necessary in retrieval algorithms for simplification, but these assumptions create uncertainties of their own. Therefore, ice cloud property retrieval uncertainties can be substantial and are often not well known. To investigate these uncertainties, reflectivity factors measured by CloudSat are compared to those calculated from particle size distributions (PSDs) to which different m-D relationships are applied. These PSDs are from data collected in situ during three flights of the Small Particles in Cirrus (SPartICus) campaign. We find that no specific habit emerges as preferred and instead we conclude that the microphysical characteristics of ice crystal populations tend to be distributed over a continuum and, therefore, cannot be categorized easily. To quantify the uncertainties in the mass-dimensional relationships, an optimal estimation inversion was run to retrieve the m-D relationship per SPartICus flight, as well as to calculate uncertainties of the m-D power law.

O2 A Band Studies for Cloud Detection and Algorithm Improvement

NASA Technical Reports Server (NTRS)

Chance, K. V.

1996-01-01

Detection of cloud parameters from space-based spectrometers can employ the vibrational bands of O2 in the (sup b1)Sigma(sub +)(sub g) yields X(sub 3) Sigma(sup -)(sub g) spin-forbidden electronic transition manifold, particularly the Delta nu = 0 A band. The GOME instrument uses the A band in the Initial Cloud Fitting Algorithm (ICFA). The work reported here consists of making substantial improvements in the line-by-line spectral database for the A band, testing whether an additional correction to the line shape function is necessary in order to correctly model the atmospheric transmission in this band, and calculating prototype cloud and ground template spectra for comparison with satellite measurements.

Determinations of cloud liquid water in the tropics from the SSM/I

NASA Technical Reports Server (NTRS)

Alishouse, John C.; Swift, Calvin; Ruf, Christopher; Snyder, Sheila; Vongsathorn, Jennifer

1989-01-01

Upward-looking microwave radiometric observations were used to validate the SSM/I determinations, and also as a basis for the determination of new coefficients. Due to insufficiency of the initial four channel algorithm for cloud liquid water, the improved algorithm was derived from the CORRAD (the University of Massachusetts autocorrelation radiometer) measurements of cloud liquid water and the matching SSM/I brightness temperatures using the standard linear regression. The correlation coefficients for the possible four channel combinations, and subsequently the best and the worst combinations were determined.

Optimal Exploitation of the Temporal and Spatial Resolution of SEVIRI for the Nowcasting of Clouds

NASA Astrophysics Data System (ADS)

Sirch, Tobias; Bugliaro, Luca

2015-04-01

Optimal Exploitation of the Temporal and Spatial Resolution of SEVIRI for the Nowcasting of Clouds An algorithm was developed to forecast the development of water and ice clouds for the successive 5-120 minutes separately using satellite data from SEVIRI (Spinning Enhanced Visible and Infrared Imager) aboard Meteosat Second Generation (MSG). In order to derive cloud cover, optical thickness and cloud top height of high ice clouds "The Cirrus Optical properties derived from CALIOP and SEVIRI during day and night" (COCS, Kox et al. [2014]) algorithm is applied. For the determination of the liquid water clouds the APICS ("Algorithm for the Physical Investigation of Clouds with SEVIRI", Bugliaro e al. [2011]) cloud algorithm is used, which provides cloud cover, optical thickness and effective radius. The forecast rests upon an optical flow method determining a motion vector field from two satellite images [Zinner et al., 2008.] With the aim of determining the ideal time separation of the satellite images that are used for the determination of the cloud motion vector field for every forecast horizon time the potential of the better temporal resolution of the Meteosat Rapid Scan Service (5 instead of 15 minutes repetition rate) has been investigated. Therefore for the period from March to June 2013 forecasts up to 4 hours in time steps of 5 min based on images separated by a time interval of 5 min, 10 min, 15 min, 30 min have been created. The results show that Rapid Scan data produces a small reduction of errors for a forecast horizon up to 30 minutes. For the following time steps forecasts generated with a time interval of 15 min should be used and for forecasts up to several hours computations with a time interval of 30 min provide the best results. For a better spatial resolution the HRV channel (High Resolution Visible, 1km instead of 3km maximum spatial resolution at the subsatellite point) has been integrated into the forecast. To detect clouds the difference of the measured albedo from SEVIRI and the clear-sky albedo provided by MODIS has been used and additionally the temporal development of this quantity. A pre-requisite for this work was an adjustment of the geolocation accuracy for MSG and MODIS by shifting the MODIS data and quantifying the correlation between both data sets.

Cloud field classification based upon high spatial resolution textural features. I - Gray level co-occurrence matrix approach

NASA Technical Reports Server (NTRS)

Welch, R. M.; Sengupta, S. K.; Chen, D. W.

1988-01-01

Stratocumulus, cumulus, and cirrus clouds were identified on the basis of cloud textural features which were derived from a single high-resolution Landsat MSS NIR channel using a stepwise linear discriminant analysis. It is shown that, using this method, it is possible to distinguish high cirrus clouds from low clouds with high accuracy on the basis of spatial brightness patterns. The largest probability of misclassification is associated with confusion between the stratocumulus breakup regions and the fair-weather cumulus.