The great minds at NASA's Jet Propulsion Laboratory have always known that by analyzing what they've already accomplished — in data, documents, videos and images — they could save months of time and millions of dollars, and even enable breakthrough discoveries. However, every mission has its own unique data sets and tools are constantly evolving, creating challenges for engineers and scientists who want to quickly search and analyze petabytes of disparate data that in some cases had been created decades apart.
Technology has finally caught up to JPL's dream. Its interactive search and analytics project uses open development technologies, methodologies and tools on cloud platforms to unlock the data.
"It has never been accessible before — the ability to search data across all these different types of domains and find something quickly," says Tom Soderstrom, IT chief technology and innovation officer.
Now scientists can search and analyze not only data from previous missions, but also data from active missions. For instance, data from the Mars Rover Curiosity, which landed on Mars in August 2012 and is still in use, can be copied and sent to a secure cloud-based system where analytics can be applied to it without risking the mission. Scientists can also analyze data that comes directly from space — some 20 billion data points.
The project isn't finished, but JPL staff already knows it has limitless potential. The new system has already saved dozens of man-hours by automating previously manual processes. "That's important because the amount of data created by a typical mission is staggering," says Daniel Zayas, thermal operations lead for Curiosity. "Data sets are only going to get bigger."