Nanotechnology Now

Our NanoNews Digest Sponsors





Heifer International

Wikipedia Affiliate Button


android tablet pc

Home > Press > New tool enables powerful data analysis

This image of an early moment in the simulated mixing of two fluids was created by researchers using a powerful new algorithm they developed to extract features and patterns from massive data sets. In the image, blue and red spheres and the lines between them represent the branching of pockets of fluid.

Credit: Attila Gyulassi/UC Davis copyright UC Regents
This image of an early moment in the simulated mixing of two fluids was created by researchers using a powerful new algorithm they developed to extract features and patterns from massive data sets. In the image, blue and red spheres and the lines between them represent the branching of pockets of fluid.

Credit: Attila Gyulassi/UC Davis copyright UC Regents

Abstract:
A powerful yet compact algorithm has been developed that can be used on laptop computers to extract features and patterns from huge and complex data sets

New tool enables powerful data analysis

Davis, CA | Posted on January 8th, 2009

A powerful computing tool that allows scientists to extract features and patterns from enormously large and complex sets of raw data has been developed by scientists at University of California, Davis, and Lawrence Livermore National Laboratory. The tool - a set of problem-solving calculations known as an algorithm - is compact enough to run on computers with as little as two gigabytes of memory.

The team that developed this algorithm has already used it to probe a slew of phenomena represented by billions of data points, including analyzing and creating images of flame surfaces; searching for clusters and voids in a virtual universe experiment; and identifying and tracking pockets of fluid in a simulated mixing of two fluids.

"What we've developed is a workable system of handling any data in any dimension," said Attila Gyulassy, who led the five-year development effort while pursuing a PhD in computer science at UC Davis. "We expect this algorithm will become an integral part of a scientist's toolbox to answer questions about data."

A paper describing the new algorithm was published in the November-December issue of IEEE Transactions on Visualization and Computer Graphics.

Computers are widely used to perform simulations of real-world phenomena and to capture results of physical experiments and observations, storing this information as collections of numbers. But as the size of these data sets has burgeoned, hand-in-hand with computer capacity, analysis has grown increasingly difficult.

A mathematical tool to extract and visualize useful features from data sets has existed for nearly 40 years - in theory. Called the Morse-Smale complex, it partitions sets by similarity of features and encodes them into mathematical terms. But working with the Morse-Smale complex is not easy. "It's a powerful language. But a cost of that, is that using it meaningfully for practical applications is very difficult," Gyulassy said.

Gyulassy's algorithm divides data sets into parcels of cells, then analyzes each parcel separately using the Morse-Smale complex. Results of those computations are then merged together. As new parcels are created from merged parcels, they are analyzed and merged yet again. At each step, data that do not need to be stored in memory are discarded, drastically reducing the computing power required to run the calculations.

One of Gyulassy's tests of the algorithm was to use it to analyze and track the formation and movement of pockets of fluid in the simulated mixing of two fluids: one dense, one light. The complexity of this data set is so vast - it consists of more than one billion data points on a three-dimensional grid - it challenges even supercomputers, Gyulassy said. Yet the new algorithm with its streamlining features was able to perform the analysis on a laptop computer with just two gigabytes of memory. Although Gyulassy had to wait nearly 24 hours for the little machine to complete its calculations, at the end of this process he could pull up images in mere seconds to illustrate phenomena he was interested in, such as the branching of fluid pockets in the mixture.

Two main factors are driving the need for analysis of large data sets, said co-author Bernd Hamann: a surge in the use of powerful computers that can produce huge amounts of data, and an upswing in affordability and availability of sensing devices that researchers deploy in the field and lab to collect a profusion of data.

"Our data files are becoming larger and larger, while the scientist has less and less time to understand them," said Hamann, a professor of computer science and associate vice chancellor for research at UC Davis. "But what are the data good for if we don't have the means of applying mathematically sound and computationally efficient computer analysis tools to look for what is captured in them?"

Gyulassy is currently developing software that will allow others to put the algorithm to use. He expects the learning curve to be steep for this open-source product, "but if you just learn the minimal amount about what a Morse-Smale complex is," he said, "it will be pretty intuitive."

Other authors of the paper are Valerio Pascucci, who was an adjunct professor of computer science at UC Davis and a computer scientist and project leader at Lawrence Livermore National Laboratory when he did the work (he is now at University of Utah); and Peer-Timo Bremer, a computer scientist at Lawrence Livermore National Laboratory.

The research was supported in part by the National Science Foundation. Gyulassy also received support from the Lawrence Scholar Program.

####

For more information, please click here

Contacts:
Liese Greensfelder

530-752-6101

Copyright © University of California - Davis

If you have a comment, please Contact us.

Issuers of news releases, not 7th Wave, Inc. or Nanotechnology Now, are solely responsible for the accuracy of the content.

Bookmark:
Delicious Digg Newsvine Google Yahoo Reddit Magnoliacom Furl Facebook

Related News Press

News and information

Silicene Labs Announces the Launch of Patent-Pending, 2D Materials Composite Index™ : The Initial 2D Materials Composite Index™ for Q2 2014 Is: 857.3; Founders Include World-Renowned Physicist and Seasoned Business and IP Professionals July 24th, 2014

Iranian Scientists Produce Transparent Nanocomposite Coatings with Longer Lifetime July 24th, 2014

Deadline Announced for Registration in 7th Int'l Nanotechnology Festival in Iran July 23rd, 2014

A Crystal Wedding in the Nanocosmos July 23rd, 2014

Software

Physicists Use Computer Models to Reveal Quantum Effects in Biological Oxygen Transport: The team solved a long-standing question by explaining why oxygen – and not deadly carbon monoxide – preferably binds to the proteins that transport it around the body. July 17th, 2014

New computer program aims to teach itself everything about anything June 12th, 2014

Lumerical files a provisional patent that extends the standard eigenmode expansion propagation technique to better address waveguide component design. Lumerical’s EME propagation tool will address a wide set of waveguide applications in silicon photonics and integrated optics April 16th, 2014

New Integrated Raman Spectroscopy & Imaging Software from CRAIC Technologies: CRAIC Technologies introduces Lambdafire-R™ integrated Raman microspectroscopy & imaging software for Windows 8 ® March 26th, 2014

Announcements

Silicene Labs Announces the Launch of Patent-Pending, 2D Materials Composite Index™ : The Initial 2D Materials Composite Index™ for Q2 2014 Is: 857.3; Founders Include World-Renowned Physicist and Seasoned Business and IP Professionals July 24th, 2014

Iranian Scientists Produce Transparent Nanocomposite Coatings with Longer Lifetime July 24th, 2014

Deadline Announced for Registration in 7th Int'l Nanotechnology Festival in Iran July 23rd, 2014

A Crystal Wedding in the Nanocosmos July 23rd, 2014

Tools

Hysitron is Awarded TWO R&D 100 Awards for Highly Innovative Technology Developments in the Areas of Extreme Environments and Biological Mechanical Property Testing July 23rd, 2014

EPFL Research on the use of AFM based nanoscale IR spectroscopy for the study of single amyloid molecules wins poster competition at Swiss Physics Society meeting July 22nd, 2014

The Hiden EQP Plasma Diagnostic with on-board MCA July 22nd, 2014

Nanometrics Announces Upcoming Investor Events July 22nd, 2014

NanoNews-Digest
The latest news from around the world, FREE



  Premium Products
NanoNews-Custom
Only the news you want to read!
 Learn More
NanoTech-Transfer
University Technology Transfer & Patents
 Learn More
NanoStrategies
Full-service, expert consulting
 Learn More














ASP
Nanotechnology Now Featured Books




NNN

The Hunger Project







© Copyright 1999-2014 7th Wave, Inc. All Rights Reserved PRIVACY POLICY :: CONTACT US :: STATS :: SITE MAP :: ADVERTISE